Commit graph

483 commits

Author SHA1 Message Date
Michael Eischer
046b0e711d repository: add SaveBlobAsync method 2025-11-26 21:18:21 +01:00
Michael Eischer
07d090f233 repository: expose AssociatedBlobSet via repository interface 2025-11-26 20:59:08 +01:00
Michael Eischer
0f05277b47 index: add sub and intersect method to AssociatedSet 2025-11-26 20:59:08 +01:00
Michael Eischer
cf409b7c66 automatically batch snapshots in copy 2025-11-23 17:40:37 +01:00
Michael Eischer
405813f250 repository: fix LookupBlobSize to also report pending blobs 2025-11-23 17:09:07 +01:00
Michael Eischer
81d8bc4ade repository: replace CopyBlobs with Repack implementation 2025-11-23 16:06:29 +01:00
Winfried Plappert
b24b088978 restic copy --batch: The mighty linter
I cave in - no double comment
2025-11-19 07:34:39 +00:00
Winfried Plappert
fc3de018bc restic copy --batch - fussy linter
internal/repository/repack.go: I have to please the mighty linter.
2025-11-19 07:29:09 +00:00
Winfried Plappert
b87f7586e4 restic copy --batch: a fresh start from commit 382616747
Instead of rebasing my code, I decided to start fresh, since WithBlobUploader()
has been introduced.

changelog/unreleased/issue-5453:
doc/045_working_with_repos.rst:
the usual

cmd/restic/cmd_copy.go:
gather all snaps to be collected - collectAllSnapshots()
run overall copy step - func copyTreeBatched()
helper copySaveSnapshot() to save the corresponding snapshot

internal/repository/repack.go:
introduce wrapper CopyBlobs(), which passes parameter `uploader restic.BlobSaver` from
WithBlobUploader() via copyTreeBatched() to repack().

internal/backend/local/local_windows.go:
I did not touch it, but gofmt did: whitespace
2025-11-19 07:09:24 +00:00
Michael Eischer
14f3bc8232
Merge pull request #5560 from MichaelEischer/index-iterators
index: port to  modern Go iterators
2025-11-16 13:24:48 +01:00
Michael Eischer
b587c126e0 Fix linter warning 2025-11-16 12:56:37 +01:00
Michael Eischer
9944ef7a7c index: convert AssociatedSet to go iterators 2025-11-16 12:56:37 +01:00
Michael Eischer
38c543457e index: convert to implement modern go iterators 2025-11-16 12:56:37 +01:00
Michael Eischer
a0925fa922 repository: set progress bar maximum in Repack 2025-11-16 12:51:46 +01:00
Michael Eischer
b2afccbd96 repository: remove unused obsoletePacks return values from Repack 2025-11-16 12:51:46 +01:00
Michael Eischer
0624b656b8
Merge pull request #5558 from MichaelEischer/simplify-blob-upload
repository: enforce correct usage of SaveBlob
2025-11-16 12:51:01 +01:00
Michael Eischer
c6e33c3954 repository: enforce that SaveBlob is called within WithBlobUploader
This is achieved by removing SaveBlob from the public API and only
returning it via a uploader object that is passed in by
WithBlobUploader.
2025-10-12 18:26:26 +02:00
Michael Eischer
ac4642b479 repository: replace StartPackUploader+Flush with WithBlobUploader
The new method combines both step into a single wrapper function. Thus
it ensures that both are always called in pairs. As an additional
benefit this slightly reduces the boilerplate to upload blobs.
2025-10-08 22:49:45 +02:00
Michael Eischer
b7bbb408ee check: refactor pack selection for read data
Drop the `packs` map from the internal state of the checker. Instead the
Packs(...) method now calls a filter callback that can select the
packs intended for checking.
2025-10-03 23:45:05 +02:00
Michael Eischer
4426dfe6a9 repository: replace SetIndex method with internal loadIndexWithCallback method 2025-10-03 19:36:57 +02:00
Michael Eischer
f0955fa931 repository: add Checker() method to repository to replace unchecked cast 2025-10-03 19:34:33 +02:00
Michael Eischer
189b295c30 repository: add dedicated test helper 2025-10-03 19:34:33 +02:00
Michael Eischer
82971ad7f0 check: split index/pack check into repository package 2025-10-03 19:34:32 +02:00
Michael Eischer
56ac8360c7 data: split node and snapshot code from restic package 2025-10-03 19:10:39 +02:00
Michael Eischer
52eb66929f repository: deduplicate index progress bar initializaton 2025-10-03 18:55:46 +02:00
Michael Eischer
b6c50662da repository: don't ignore cache clearing error 2025-10-03 18:22:42 +02:00
Michael Eischer
4dc71f24c5 backends: pass error logger to backends 2025-10-03 18:22:42 +02:00
Michael Eischer
1c7bb15327
Merge pull request #5451 from greatroar/concurrency
Concurrency simplifications
2025-09-24 22:22:40 +02:00
Michael Eischer
4edfd36c8f
Merge pull request #5363 from zmanda/fix-gh-5258-backup-exits-with-wrong-code-on-ctrl-c
bugfix: fatal errors do not keep underlying error
2025-09-24 22:04:38 +02:00
Michael Eischer
88bdf20bd8 Reduce linter ignores 2025-09-21 22:24:27 +02:00
Michael Eischer
60d80a6127 Fix linter warnings 2025-09-21 22:24:15 +02:00
Srigovind Nayak
ce089f7e2d
errors: standardize error wrapping for Fatal errors
* replace all occurences of  `errors.Fatal(err.Error())` with `errors.Fatalf("%s", err)` so that the error wrapping is correct across the codebase

* updated the review comments
2025-09-13 23:32:40 +05:30
Michael Eischer
de29d74707 check: fix error reporting on download retry 2025-09-08 11:45:28 +02:00
greatroar
2c39b1f84f internal/repository/index: Simplify MasterIndex concurrency 2025-07-18 15:06:37 +02:00
Srigovind Nayak
f13e9c10a4
Add support for additional compression levels fastest and better (#5321)
* repository: expose addtional compression levels

* adding better and fastest compression levels for zstd

* repository: add changelog entry for issue-4728

* chore: fix golint issues

* chore: sort compression modes in the help text

* updating review comments
2025-03-31 21:21:12 +02:00
Michael Eischer
13cb90b83a
Merge pull request #5295 from MichaelEischer/randomize-pack-order
Randomize blob to pack file assignment
2025-03-25 18:13:49 +01:00
Michael Eischer
ccb92f5bf0 repository/packer: add unit test for Merge method 2025-03-24 17:04:02 +01:00
Michael Eischer
8d2d50d095 repository: merge small pack files before flushing
This prevents chunk size leaks when a backup only consists of a small
file which is split in two parts, which end up in two individual pack
files.
2025-03-23 12:29:16 +01:00
Michael Eischer
62453f9356 repository: randomly distribute blobs over two pack files 2025-03-23 12:29:16 +01:00
Martin Smith
6caad10840 Remove extra brackets. 2025-03-23 10:11:43 +00:00
Martin Smith
6e45c51509 Fix name including package name and variable shadowing package. 2025-03-23 10:01:19 +00:00
Martin Smith
5e7333d28d Unify repository receiver name. 2025-03-23 09:57:59 +00:00
Martin Smith
3788605127 Rename unused parameters to '_'. 2025-03-22 18:20:30 +00:00
Martin Smith
092899df8b Remove unnecessary type conversions. 2025-03-22 18:20:30 +00:00
Michael Eischer
ef692991a4
Merge pull request #5183 from wplapper/cmd_prune
restic prune: selection of packs to repack based on size
2025-03-22 15:43:32 +01:00
Michael Eischer
69249372bf
Merge pull request #5249 from MichaelEischer/fix-repair-index
Prevent creation of oversized indexes and automatically rewrite them.
2025-03-22 14:29:39 +01:00
Winfried Plappert
a2a1309fd9 prune: make small pack size configureable for prune all changes together
cmd_prune.go: added option `--repack-smaller-than`
prune.go: added field `SmallPackBytes` to `PruneOptions`, including checking and processing
prune_test.go: added test `TestPruneSmall`
doc/060_forget.rst: added description of enhancement
changelog/unreleased/issue-5109: description of enhancement
2025-02-18 16:54:44 +00:00
Michael Eischer
c970e58739 backend: refactor backend Connections and HasAtomicReplace into Properties 2025-02-16 22:27:58 +01:00
Michael Eischer
39e63ee4e3 index: add tests for oversized index handling 2025-02-16 17:42:00 +01:00
Michael Eischer
3b8d15d651 index: rewrite oversized indexes 2025-02-16 17:03:14 +01:00