Archive.delete: don't reuse msgpack Unpacker after an unpacking failure

msgpack does not support reusing an Unpacker instance after an unpacking
failure - a new instance must be created instead.

In the forced-delete path, item metadata was unpacked via "for item in
unpacker" with an "except (TypeError, ValueError)" handler. msgpack's
FormatError/StackError are ValueError subclasses, so a real unpacking
failure was caught there and the (now broken) unpacker kept being reused
for the following chunks, making them all fail too - leaking their chunk
references.

Restructure the loop to use next(unpacker) so each failure mode gets a
tight handler:
- StopIteration: buffer drained, feed next chunk
- msgpack.UnpackException: corruption; in forced mode replace the unpacker
  with a fresh instance and skip the rest of the chunk
- TypeError/ValueError: only wraps Item processing (bad types yielded)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Thomas Waldmann 2026-06-18 19:42:53 +02:00
parent 8abdd3b8bf
commit b09bbed3de
No known key found for this signature in database
GPG key ID: 243ACFA951F78E01

View file

@ -1035,22 +1035,35 @@ Utilization of max. archive size: {csize_max:.0%}
data = self.key.decrypt(items_id, data)
unpacker.feed(data)
chunk_decref(items_id, stats)
try:
for item in unpacker:
while True:
try:
item = next(unpacker)
except StopIteration:
# no more (complete) items in the buffer, feed the next chunk
break
except msgpack.UnpackException:
# items metadata corrupted. the unpacker can't be reused after an
# unpacking failure, so create a fresh one and skip the rest of this chunk.
if forced == 0:
raise
error = True
unpacker = msgpack.Unpacker(use_list=False)
break
try:
item = Item(internal_dict=item)
if 'chunks' in item:
part = not self.consider_part_files and 'part' in item
for chunk_id, size, csize in item.chunks:
chunk_decref(chunk_id, stats, part=part)
except (TypeError, ValueError):
# if items metadata spans multiple chunks and one chunk got dropped somehow,
# it could be that unpacker yields bad types
if forced == 0:
raise
error = True
except (TypeError, ValueError):
# if items metadata spans multiple chunks and one chunk got dropped somehow,
# it could be that unpacker yields bad types
if forced == 0:
raise
error = True
if progress:
pi.finish()
except (msgpack.UnpackException, Repository.ObjectNotFound):
except Repository.ObjectNotFound:
# items metadata corrupted
if forced == 0:
raise