Commit graph

2601 commits

Author SHA1 Message Date
Michael Eischer
f3a89bfff6
Merge pull request #5612 from MichaelEischer/repository-async-saveblob
repository: add async blob upload method
2025-11-26 21:34:35 +01:00
Michael Eischer
5cc8636047
Merge pull request #5614 from MichaelEischer/fix-lookupblobsize
repository: fix LookupBlobSize to also return pending blobs
2025-11-26 21:24:32 +01:00
Michael Eischer
6769d26068 archiver: improve test reliability 2025-11-26 21:21:16 +01:00
Michael Eischer
5607fd759f repository: fix race condition for blobSaver shutdown
wg.Go() may not be called after wg.Wait(). This prevents connecting two
errgroups such that the errors are propagated between them if the child
errgroup dynamically starts goroutines. Instead use just a single errgroup,
and sequence the shutdown using a sync.WaitGroup. This is far simpler
and does not require any "clever" tricks.
2025-11-26 21:18:22 +01:00
Michael Eischer
9f87e9096a repository: add tests for SaveBlobAsync 2025-11-26 21:18:22 +01:00
Michael Eischer
d8dcd6d115 archiver: add buffer test 2025-11-26 21:18:22 +01:00
Michael Eischer
3f92987974 archiver: assert number of uploaded chunks in fileSaver test 2025-11-26 21:18:22 +01:00
Michael Eischer
7f6fdcc52c archiver: convert buffer pool to use sync.Pool 2025-11-26 21:18:22 +01:00
Michael Eischer
dd6cb0dd8e archiver: port to repository.SaveBlobAsync 2025-11-26 21:18:22 +01:00
Michael Eischer
046b0e711d repository: add SaveBlobAsync method 2025-11-26 21:18:21 +01:00
Michael Eischer
84dda4dc74 check: use AssociatedBlobSet 2025-11-26 20:59:39 +01:00
Michael Eischer
07d090f233 repository: expose AssociatedBlobSet via repository interface 2025-11-26 20:59:08 +01:00
Michael Eischer
0f05277b47 index: add sub and intersect method to AssociatedSet 2025-11-26 20:59:08 +01:00
Michael Eischer
7e80536a9b
Merge pull request #5472 from wplapper/cmd_copy_stream
restic copy --stream: run one large copy operation crossing snapshot boundaries - issue #5453
2025-11-26 20:57:46 +01:00
Michael Eischer
f1aabdd293 index: add test for pending blobs 2025-11-23 18:08:56 +01:00
Michael Eischer
50d376c543 repository: fix LookupBlobSize to also report pending blobs 2025-11-23 17:55:13 +01:00
Michael Eischer
cf409b7c66 automatically batch snapshots in copy 2025-11-23 17:40:37 +01:00
Michael Eischer
405813f250 repository: fix LookupBlobSize to also report pending blobs 2025-11-23 17:09:07 +01:00
Michael Eischer
81d8bc4ade repository: replace CopyBlobs with Repack implementation 2025-11-23 16:06:29 +01:00
Winfried Plappert
b24b088978 restic copy --batch: The mighty linter
I cave in - no double comment
2025-11-19 07:34:39 +00:00
Winfried Plappert
fc3de018bc restic copy --batch - fussy linter
internal/repository/repack.go: I have to please the mighty linter.
2025-11-19 07:29:09 +00:00
Winfried Plappert
b87f7586e4 restic copy --batch: a fresh start from commit 382616747
Instead of rebasing my code, I decided to start fresh, since WithBlobUploader()
has been introduced.

changelog/unreleased/issue-5453:
doc/045_working_with_repos.rst:
the usual

cmd/restic/cmd_copy.go:
gather all snaps to be collected - collectAllSnapshots()
run overall copy step - func copyTreeBatched()
helper copySaveSnapshot() to save the corresponding snapshot

internal/repository/repack.go:
introduce wrapper CopyBlobs(), which passes parameter `uploader restic.BlobSaver` from
WithBlobUploader() via copyTreeBatched() to repack().

internal/backend/local/local_windows.go:
I did not touch it, but gofmt did: whitespace
2025-11-19 07:09:24 +00:00
Michael Eischer
132f2f8a23
Merge pull request #5602 from MichaelEischer/fix-flaky-rclone-test
rclone: fix rare test failure if rclone cannot be started
2025-11-17 22:04:10 +01:00
Michael Eischer
a519d1e8df
Merge pull request #5603 from MichaelEischer/debug-flaky-windows-test
restore: enable debug logging for flaky windows test
2025-11-17 22:03:38 +01:00
Paulo Saraiva
c1a89d5150
Allow for a personal token to be specified for self-updates (#5568)
* Allow for a personal token to be specified for self-updates

This change will allow for setting the $GITHUB_ACCESS_TOKEN environment variable with a Github personal access token, allowing e.g. for higher rate limits

* Refactor github request and add test

---------

Co-authored-by: Paulo Saraiva <pauloman@cern.ch>
2025-11-17 21:39:39 +01:00
Michael Eischer
3826167474
Merge pull request #5424 from Crazycatz00/sebackup-fixes
Some checks are pending
Create and publish a Docker image / build-and-push-image (push) Waiting to run
Create and publish a Docker image / provenance (push) Blocked by required conditions
test / Linux Go 1.23.x (push) Waiting to run
test / Linux Go 1.24.x (push) Waiting to run
test / Linux (race) Go 1.25.x (push) Waiting to run
test / Windows Go 1.25.x (push) Waiting to run
test / macOS Go 1.25.x (push) Waiting to run
test / Linux Go 1.25.x (push) Waiting to run
test / Cross Compile for subset 0/3 (push) Waiting to run
test / Cross Compile for subset 1/3 (push) Waiting to run
test / Cross Compile for subset 2/3 (push) Waiting to run
test / lint (push) Waiting to run
test / Analyze results (push) Blocked by required conditions
test / docker (push) Waiting to run
Windows Backup Privilege Tweaks
2025-11-16 21:35:35 +01:00
Michael Eischer
98f56d8ada restore: enable debug logging for flaky windows test 2025-11-16 20:24:19 +01:00
Michael Eischer
1caeb2aa4d rclone: fix rare test failure if rclone cannot be started 2025-11-16 20:14:21 +01:00
crazycatz00
3ab68d4d11 fs: Clarified documentation 2025-11-16 11:53:13 -05:00
Michael Eischer
3b854d9c04
Merge pull request #5449 from provokateurin/restore-ownership-by-name
Some checks are pending
Create and publish a Docker image / build-and-push-image (push) Waiting to run
Create and publish a Docker image / provenance (push) Blocked by required conditions
test / Linux Go 1.23.x (push) Waiting to run
test / Linux Go 1.24.x (push) Waiting to run
test / Linux (race) Go 1.25.x (push) Waiting to run
test / Windows Go 1.25.x (push) Waiting to run
test / macOS Go 1.25.x (push) Waiting to run
test / Linux Go 1.25.x (push) Waiting to run
test / Cross Compile for subset 0/3 (push) Waiting to run
test / Cross Compile for subset 1/3 (push) Waiting to run
test / Cross Compile for subset 2/3 (push) Waiting to run
test / lint (push) Waiting to run
test / Analyze results (push) Blocked by required conditions
test / docker (push) Waiting to run
feat(internal/fs/node): Restore ownership by name
2025-11-16 16:50:36 +01:00
provokateurin
8fae46011a
feat(internal/fs/node): Restore ownership by name 2025-11-16 16:40:58 +01:00
Michael Eischer
c854338ad1
Merge pull request #5596 from mikix/chmod-again
backend/local: fix "operation not supported" when unlocking
2025-11-16 14:25:04 +01:00
Michael Terry
7f3e3b77ce backend/local: fix "operation not supported" when unlocking
If the repo is on a mounted folder that doesn't support chmod (like
SMB), it was causing an "operation not supported" error when trying to
chmod 666 a file before deleting it.

But it isn't generally needed before deleting a file (the folder
permissions matter there, not the file permissions). So, just drop it.
2025-11-16 08:09:51 -05:00
Michael Eischer
14f3bc8232
Merge pull request #5560 from MichaelEischer/index-iterators
index: port to  modern Go iterators
2025-11-16 13:24:48 +01:00
Michael Eischer
b587c126e0 Fix linter warning 2025-11-16 12:56:37 +01:00
Michael Eischer
9944ef7a7c index: convert AssociatedSet to go iterators 2025-11-16 12:56:37 +01:00
Michael Eischer
38c543457e index: convert to implement modern go iterators 2025-11-16 12:56:37 +01:00
Michael Eischer
393e49fc89 repository: update comment 2025-11-16 12:51:46 +01:00
Michael Eischer
a0925fa922 repository: set progress bar maximum in Repack 2025-11-16 12:51:46 +01:00
Michael Eischer
b2afccbd96 repository: remove unused obsoletePacks return values from Repack 2025-11-16 12:51:46 +01:00
Michael Eischer
0624b656b8
Merge pull request #5558 from MichaelEischer/simplify-blob-upload
repository: enforce correct usage of SaveBlob
2025-11-16 12:51:01 +01:00
Michael Eischer
157f174dd9
Merge pull request #5370 from hashier/feat/exclude-macOS-cloud-files
feat(backup): add possibility to exclude macOS cloud-only files
2025-11-16 11:57:37 +01:00
crazycatz00
d14823eb81 fs: Attempt to enable file system privileges on initialization.
Add tests to verify privileges' effects.
2025-11-07 19:31:59 -05:00
crazycatz00
01bf8977e7 fs: Use backup privileges when reading extended attributes for files too. 2025-11-07 19:31:57 -05:00
Michael Eischer
71432c7f4b
Merge pull request #5555 from MichaelEischer/extract-globaloptions
Some checks failed
Create and publish a Docker image / build-and-push-image (push) Has been cancelled
test / Linux Go 1.23.x (push) Has been cancelled
test / Linux Go 1.24.x (push) Has been cancelled
test / Linux (race) Go 1.25.x (push) Has been cancelled
test / Windows Go 1.25.x (push) Has been cancelled
test / macOS Go 1.25.x (push) Has been cancelled
test / Linux Go 1.25.x (push) Has been cancelled
test / Cross Compile for subset 0/3 (push) Has been cancelled
test / Cross Compile for subset 1/3 (push) Has been cancelled
test / Cross Compile for subset 2/3 (push) Has been cancelled
test / lint (push) Has been cancelled
test / docker (push) Has been cancelled
Create and publish a Docker image / provenance (push) Has been cancelled
test / Analyze results (push) Has been cancelled
Split globalOptions into separate package
2025-10-12 18:31:44 +02:00
Michael Eischer
c6e33c3954 repository: enforce that SaveBlob is called within WithBlobUploader
This is achieved by removing SaveBlob from the public API and only
returning it via a uploader object that is passed in by
WithBlobUploader.
2025-10-12 18:26:26 +02:00
Michael Eischer
1ef785daa3
Merge pull request #5544 from zmanda/fix-gh-5531-azure-backend-upgrade-service-version
azure: use PutBlob API for uploads instead of PutBlock API + PutBlock List API
2025-10-12 18:24:33 +02:00
Michael Eischer
aa0fb0210a
Merge pull request #5556 from greatroar/cleanup
ui/backup: Prepend, then sort (micro-optimization)
2025-10-12 18:22:36 +02:00
Michael Eischer
b6aef592f5 global: split CreateRepository and OpenRepository into smaller functions 2025-10-12 18:20:45 +02:00
Michael Eischer
588c40aaef global: unexport ReadPassword and ReadRepo 2025-10-12 18:08:26 +02:00