Commit graph

281 commits

Author SHA1 Message Date
Michael Eischer
5607fd759f repository: fix race condition for blobSaver shutdown
wg.Go() may not be called after wg.Wait(). This prevents connecting two
errgroups such that the errors are propagated between them if the child
errgroup dynamically starts goroutines. Instead use just a single errgroup,
and sequence the shutdown using a sync.WaitGroup. This is far simpler
and does not require any "clever" tricks.
2025-11-26 21:18:22 +01:00
Michael Eischer
d8dcd6d115 archiver: add buffer test 2025-11-26 21:18:22 +01:00
Michael Eischer
3f92987974 archiver: assert number of uploaded chunks in fileSaver test 2025-11-26 21:18:22 +01:00
Michael Eischer
7f6fdcc52c archiver: convert buffer pool to use sync.Pool 2025-11-26 21:18:22 +01:00
Michael Eischer
dd6cb0dd8e archiver: port to repository.SaveBlobAsync 2025-11-26 21:18:22 +01:00
Michael Eischer
046b0e711d repository: add SaveBlobAsync method 2025-11-26 21:18:21 +01:00
Michael Eischer
c6e33c3954 repository: enforce that SaveBlob is called within WithBlobUploader
This is achieved by removing SaveBlob from the public API and only
returning it via a uploader object that is passed in by
WithBlobUploader.
2025-10-12 18:26:26 +02:00
Michael Eischer
ac4642b479 repository: replace StartPackUploader+Flush with WithBlobUploader
The new method combines both step into a single wrapper function. Thus
it ensures that both are always called in pairs. As an additional
benefit this slightly reduces the boilerplate to upload blobs.
2025-10-08 22:49:45 +02:00
Michael Eischer
f0955fa931 repository: add Checker() method to repository to replace unchecked cast 2025-10-03 19:34:33 +02:00
Michael Eischer
189b295c30 repository: add dedicated test helper 2025-10-03 19:34:33 +02:00
Michael Eischer
56ac8360c7 data: split node and snapshot code from restic package 2025-10-03 19:10:39 +02:00
Michael Eischer
df7924f4df node: report error on xattr retrieval using standard error logging 2025-10-03 18:55:46 +02:00
dmotte
9017fefddd internal/archiver: fixed BackupEnd when SkipIfUnchanged is true 2025-09-23 03:07:30 +02:00
Michael Eischer
3f7121e180 backup: adapt test to changed error message 2025-09-21 22:59:59 +02:00
Michael Eischer
01bc60e96f backup: fix test on windows 2025-09-05 19:35:25 +02:00
Michael Eischer
350f6452e7 backup: test that parent directory errors can be correctly filtered 2025-09-05 19:28:18 +02:00
Michael Eischer
484cdf12e5 backup: test that missing parent directory is correctly handled 2025-09-05 19:28:18 +02:00
Michael Eischer
c8bb7bd312 backup: do not fail backup is some parent folder is inaccessible
Handle errors for parent directories of backup directories in the same
way as all other file access errors during a backup.
2025-09-05 19:28:18 +02:00
Michael Eischer
a8ce2e45cc backup: do not crash if nodeFromFileInfo fails
this could crash in two cases:
- if a directory is deleted between restic stating it and trying to list
  its directory content.
- when restic tries to list the parent directory of a backup target, but
  the parent directory has been deleted.

return an error in this case instead.
2025-09-05 19:28:18 +02:00
Michael Eischer
9f39e8a1d3 fs/reader: return proper error on invalid filename 2025-04-11 22:07:31 +02:00
Michael Eischer
3a995172b7 fs: rewrite Reader to build fs tree up front
This adds proper support for filenames that include directories. For
example, `/foo/bar` would result in an error when trying to open `/foo`.

The directory tree is now build upfront. This ensures let's the
directory tree construction be handled only once. All accessors then
only have to look up the constructed directory entries.
2025-04-11 21:37:40 +02:00
Martin Smith
a389977bd7 Remove redudnant error check, handled above. 2025-03-23 10:05:13 +00:00
Martin Smith
3788605127 Rename unused parameters to '_'. 2025-03-22 18:20:30 +00:00
Martin Smith
092899df8b Remove unnecessary type conversions. 2025-03-22 18:20:30 +00:00
Michael Eischer
99e105eeb6 repository: restrict SaveUnpacked and RemoveUnpacked
Those methods now only allow modifying snapshots. Internal data types
used by the repository are now read-only. The repository-internal code
can bypass the restrictions by wrapping the repository in an
`internalRepository` type.

The restriction itself is implemented by using a new datatype
WriteableFileType in the SaveUnpacked and RemoveUnpacked methods. This
statically ensures that code cannot bypass the access restrictions.

The test changes are somewhat noisy as some of them modify repository
internals and therefore require some way to bypass the access
restrictions. This works by capturing an `internalRepository` or
`Backend` when creating the Repository using a test helper function.
2025-01-13 22:39:57 +01:00
Michael Wildman
da71e77b28 backup: allow excluding online-only cloud files 2025-01-13 21:11:23 +01:00
Michael Eischer
9a99141a5f fs: remove os.FileInfo from fs.ExtendedFileInfo
Only the `Sys()` value from os.FileInfo is kept as field `sys` to
support Windows. The os.FileInfo removal ensures that for values like
`ModTime` that existed in both data structures there's no more confusion
which value is actually used.
2024-11-30 17:07:36 +01:00
Michael Eischer
847b2efba2 archiver: remove fs parameter from fileChanged function 2024-11-30 16:19:16 +01:00
Michael Eischer
641390103d fs: inline ExtendedStat 2024-11-30 16:19:16 +01:00
Michael Eischer
c5fb46da53 archiver: ignore files removed in the meantime 2024-11-30 15:30:42 +01:00
Michael Eischer
6cb19e0190 archiver: fix file type change test
The test did not test the case that the type of a file changed
unexpectedly.
2024-11-16 16:09:17 +01:00
Michael Eischer
48dbefc37e fs / archiver: convert to handle based interface
The actual implementation still relies on file paths, but with the
abstraction layer in place, an FS implementation can ensure atomic file
accesses in the future.
2024-11-16 12:56:23 +01:00
Michael Eischer
2f2ce9add2 fs: remove Stat from FS interface 2024-11-16 12:56:23 +01:00
Michael Eischer
623ba92b98 fs: drop unused permission parameter from OpenFile 2024-11-16 12:56:23 +01:00
Michael Eischer
f8031561f2 archiver: deduplicate error filtering 2024-11-15 17:58:06 +01:00
Michael Eischer
e65f4e2231 backup: include start and end time in json output
The timestamps were already stored in the created snapshot.
2024-11-01 16:31:34 +01:00
Michael Eischer
26b77a543d archiver: use correct filepath in fileSaver for vss
When using the VSS FS, then `f.Name()` contained the filename in the
snapshot. This caused a double mapping when calling NodeFromFileInfo.
2024-10-18 21:41:02 +02:00
Michael Eischer
75711446e1 fs: move NodeFromFileInfo into FS interface 2024-10-16 21:17:21 +02:00
Michael Eischer
f44b7cdf8c backup: exclude irregular files from backup
restic cannot backup irregular files as those don't behave like normal
files. Thus skip them with an error.
2024-10-16 20:54:08 +02:00
Michael Eischer
cc7f99125a minimize usage of internal/fs in tests 2024-08-31 18:20:41 +02:00
Michael Eischer
ca1e5e10b6 add proper constants for node type 2024-08-31 18:20:01 +02:00
Michael Eischer
7bb92dc7bd archiver: use ExtendedStat from FS interface
With this change, NodeFromFileInfo is the last function that bypasses
the FS interface in the archiver.
2024-08-31 18:05:09 +02:00
Michael Eischer
70fbad6623 archiver: minimize imports 2024-08-31 18:04:37 +02:00
Michael Eischer
6fd5d5f2d5 archiver: move helper functions to combine rejects 2024-08-31 18:04:22 +02:00
Michael Eischer
f1585af0f2 move include/exclude options to filter package 2024-08-31 18:04:07 +02:00
Michael Eischer
41c031a19e backup: move RejectFuncs to archiver package 2024-08-31 18:03:35 +02:00
Michael Eischer
f9dbcd2531 backup: convert reject funcs to use FS interface
Depending on parameters the paths in a snapshot do not directly
correspond to real paths on the filesystem. Therefore, reject funcs must
use the FS interface to work correctly.
2024-08-31 18:03:02 +02:00
Michael Eischer
c6fae0320e archiver: hide implementation details 2024-08-31 17:52:45 +02:00
Michael Eischer
507842b614 fs: remove Open method from FS interface 2024-08-31 17:37:25 +02:00
Michael Eischer
0ddb4441d7 fs: clean up helper functions 2024-08-31 17:37:25 +02:00