restic/changelog/0.13.0_2022-03-26/pull-3593

Enhancement: Improve `copy` performance by parallelizing IO

Restic copy previously only used a single thread for copying blobs between
repositories, which resulted in limited performance when copying small blobs
to/from a high latency backend (i.e. any remote backend, especially b2).

Copying will now use 8 parallel threads to increase the throughput of the copy
operation.

https://github.com/restic/restic/pull/3593
Review, reword and polish unreleased changelog entries 2022-03-25 19:15:41 -04:00			Enhancement: Improve `copy` performance by parallelizing IO
Parallelize blob upload/download for restic copy Currently restic copy will copy each blob from every snapshot serially, which has performance implications on high-latency backends such as b2. This commit introduces 8x parallelism for blob downloads/uploads which can improve restic copy operations up to 8x for repositories with many small blobs on b2. This commit also addresses the TODO comment in the copyTree function. Related work: A more thorough improvement of the restic copy performance can be found in PR #3513 2021-12-14 04:33:24 -05:00
			`Restic copy previously only used a single thread for copying blobs between`
			`repositories, which resulted in limited performance when copying small blobs`
			`to/from a high latency backend (i.e. any remote backend, especially b2).`
Review, reword and polish unreleased changelog entries 2022-03-25 19:15:41 -04:00
Parallelize blob upload/download for restic copy Currently restic copy will copy each blob from every snapshot serially, which has performance implications on high-latency backends such as b2. This commit introduces 8x parallelism for blob downloads/uploads which can improve restic copy operations up to 8x for repositories with many small blobs on b2. This commit also addresses the TODO comment in the copyTree function. Related work: A more thorough improvement of the restic copy performance can be found in PR #3513 2021-12-14 04:33:24 -05:00			`Copying will now use 8 parallel threads to increase the throughput of the copy`
			`operation.`

			`https://github.com/restic/restic/pull/3593`