Commit graph

9466 commits

Author SHA1 Message Date
gunar
7101f11133
Fail fast for invalid RESTIC_PACK_SIZE env values (#5592)
Some checks failed
Create and publish a Docker image / build-and-push-image (push) Has been cancelled
test / Linux Go 1.24.x (push) Has been cancelled
test / Linux (race) Go 1.25.x (push) Has been cancelled
test / Windows Go 1.25.x (push) Has been cancelled
test / macOS Go 1.25.x (push) Has been cancelled
test / Linux Go 1.25.x (push) Has been cancelled
test / Cross Compile for subset 0/3 (push) Has been cancelled
test / Cross Compile for subset 1/3 (push) Has been cancelled
test / Cross Compile for subset 2/3 (push) Has been cancelled
test / lint (push) Has been cancelled
test / docker (push) Has been cancelled
Create and publish a Docker image / provenance (push) Has been cancelled
test / Analyze results (push) Has been cancelled
Co-authored-by: Michael Eischer <michael.eischer@fau.de>
2026-02-01 15:45:31 +01:00
Michael Eischer
8bff5cead0
Merge pull request #5696 from restic/dependabot/go_modules/github.com/minio/minio-go/v7-7.0.98 2026-02-01 12:29:55 +01:00
Michael Eischer
5e43a44b15
Merge pull request #5680 from castilma/unlock-doc 2026-02-01 12:13:52 +01:00
Michael Eischer
67c13c643d
Merge pull request #5691 from MichaelEischer/fix-rewriter-error 2026-02-01 12:09:52 +01:00
dependabot[bot]
b706c19614
build(deps): bump github.com/minio/minio-go/v7 from 7.0.97 to 7.0.98
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go) from 7.0.97 to 7.0.98.
- [Release notes](https://github.com/minio/minio-go/releases)
- [Commits](https://github.com/minio/minio-go/compare/v7.0.97...v7.0.98)

---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
  dependency-version: 7.0.98
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-02-01 11:04:31 +00:00
Michael Eischer
da2ed89ffd
Merge pull request #5697 from restic/dependabot/go_modules/github.com/klauspost/compress-1.18.3 2026-02-01 12:01:36 +01:00
Michael Eischer
cf3793bb41
Merge pull request #5695 from restic/dependabot/go_modules/github.com/Azure/azure-sdk-for-go/sdk/storage/azblob-1.6.4 2026-02-01 12:00:39 +01:00
Michael Eischer
db8e379fd4
Merge pull request #5694 from restic/dependabot/go_modules/cloud.google.com/go/storage-1.59.2 2026-02-01 12:00:05 +01:00
Michael Eischer
4f73daa761
Merge pull request #5693 from restic/dependabot/go_modules/golang-x-deps-173d0ad829 2026-02-01 11:59:31 +01:00
Michael Eischer
48cfa908ed
Merge pull request #5692 from restic/dependabot/github_actions/docker/login-action-3.7.0 2026-02-01 11:56:38 +01:00
Michael Eischer
d3c225627f
Merge pull request #5682 from wplapper/docs_list 2026-02-01 11:55:37 +01:00
Michael Eischer
07d380d54b
Merge pull request #5191 from wplapper/cmd_rewrite_include 2026-02-01 11:53:05 +01:00
Winfried Plappert
b544e71cac restic list doc - documenmtation
fixed wording for paragraph and inconsistent underlining
2026-02-01 06:12:56 +00:00
Winfried Plappert
099650f883 docs: restic list
corrected typo
2026-02-01 06:02:39 +00:00
Winfried Plappert
6154685c3a docs: add documentation for restic list
added a file doc/view_repository.rst which contains the
description of `restic list ...`
2026-02-01 06:02:39 +00:00
dependabot[bot]
66bb196591
build(deps): bump github.com/klauspost/compress from 1.18.2 to 1.18.3
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress) from 1.18.2 to 1.18.3.
- [Release notes](https://github.com/klauspost/compress/releases)
- [Commits](https://github.com/klauspost/compress/compare/v1.18.2...v1.18.3)

---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
  dependency-version: 1.18.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-02-01 01:43:15 +00:00
dependabot[bot]
2be17d2313
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
Bumps [github.com/Azure/azure-sdk-for-go/sdk/storage/azblob](https://github.com/Azure/azure-sdk-for-go) from 1.6.3 to 1.6.4.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/storage/azblob/v1.6.3...sdk/storage/azblob/v1.6.4)

---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
  dependency-version: 1.6.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-02-01 01:42:57 +00:00
dependabot[bot]
34ba097162
build(deps): bump cloud.google.com/go/storage from 1.58.0 to 1.59.2
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) from 1.58.0 to 1.59.2.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/spanner/v1.58.0...storage/v1.59.2)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
  dependency-version: 1.59.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-02-01 01:42:48 +00:00
dependabot[bot]
38f1fb61f3
build(deps): bump the golang-x-deps group with 5 updates
Bumps the golang-x-deps group with 5 updates:

| Package | From | To |
| --- | --- | --- |
| [golang.org/x/crypto](https://github.com/golang/crypto) | `0.46.0` | `0.47.0` |
| [golang.org/x/net](https://github.com/golang/net) | `0.48.0` | `0.49.0` |
| [golang.org/x/sys](https://github.com/golang/sys) | `0.39.0` | `0.40.0` |
| [golang.org/x/term](https://github.com/golang/term) | `0.38.0` | `0.39.0` |
| [golang.org/x/text](https://github.com/golang/text) | `0.32.0` | `0.33.0` |


Updates `golang.org/x/crypto` from 0.46.0 to 0.47.0
- [Commits](https://github.com/golang/crypto/compare/v0.46.0...v0.47.0)

Updates `golang.org/x/net` from 0.48.0 to 0.49.0
- [Commits](https://github.com/golang/net/compare/v0.48.0...v0.49.0)

Updates `golang.org/x/sys` from 0.39.0 to 0.40.0
- [Commits](https://github.com/golang/sys/compare/v0.39.0...v0.40.0)

Updates `golang.org/x/term` from 0.38.0 to 0.39.0
- [Commits](https://github.com/golang/term/compare/v0.38.0...v0.39.0)

Updates `golang.org/x/text` from 0.32.0 to 0.33.0
- [Release notes](https://github.com/golang/text/releases)
- [Commits](https://github.com/golang/text/compare/v0.32.0...v0.33.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.47.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: golang-x-deps
- dependency-name: golang.org/x/net
  dependency-version: 0.49.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: golang-x-deps
- dependency-name: golang.org/x/sys
  dependency-version: 0.40.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: golang-x-deps
- dependency-name: golang.org/x/term
  dependency-version: 0.39.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: golang-x-deps
- dependency-name: golang.org/x/text
  dependency-version: 0.33.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: golang-x-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-02-01 01:42:37 +00:00
dependabot[bot]
827c7bcae8
build(deps): bump docker/login-action from 3.6.0 to 3.7.0
Bumps [docker/login-action](https://github.com/docker/login-action) from 3.6.0 to 3.7.0.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](5e57cd1181...c94ce9fb46)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-version: 3.7.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-02-01 01:42:36 +00:00
Winfried Plappert
bcd4168428
Enhancement: calls to SnapshotFilter.FindLatest() can be simplified (#5688)
Some checks are pending
Create and publish a Docker image / build-and-push-image (push) Waiting to run
Create and publish a Docker image / provenance (push) Blocked by required conditions
test / Linux Go 1.24.x (push) Waiting to run
test / Linux (race) Go 1.25.x (push) Waiting to run
test / Windows Go 1.25.x (push) Waiting to run
test / macOS Go 1.25.x (push) Waiting to run
test / Linux Go 1.25.x (push) Waiting to run
test / Cross Compile for subset 0/3 (push) Waiting to run
test / Cross Compile for subset 1/3 (push) Waiting to run
test / Cross Compile for subset 2/3 (push) Waiting to run
test / lint (push) Waiting to run
test / Analyze results (push) Blocked by required conditions
test / docker (push) Waiting to run
2026-01-31 23:04:01 +01:00
Michael Eischer
901235efc9 rewrite: skip snapshot parts not matchable by include patterns 2026-01-31 22:42:02 +01:00
Michael Eischer
ef1d525f22 rewriter: return correct error if tree iteration fails 2026-01-31 22:07:07 +01:00
Michael Eischer
74d60ad223 rewriter: test KeepEmptyDirectory option 2026-01-31 22:01:23 +01:00
Michael Eischer
0d71f70a22 minor cleanups and typos 2026-01-31 22:01:23 +01:00
Winfried Plappert
ee154ce0ab restic rewrite --include
added function Count() to the *TreeWriter methods
2026-01-31 19:58:29 +00:00
Winfried Plappert
b6af01bb28 restic rewrite integration test
convert exclusive lock on repository to 'no-lock'
2026-01-31 19:43:03 +00:00
Winfried Plappert
5148608c39 restic rewrite include - based on restic 0.18.1
cmd/restic/cmd_rewrite.go:
introduction of include filters for this command:
- add include filters, add error checking code
- add new parameter 'keepEmptyDirectoryFunc' to 'walker.NewSnapshotSizeRewriter()',
  so empty directories have to be kept to keep the directory structure intact
- add parameter 'keepEmptySnapshot' to 'filterAndReplaceSnapshot()' to keep snapshots
  intact when nothing is to be included
- introduce helper function 'gatherIncludeFilters()' and 'gatherExcludeFilters()' to
  keep code flow clean

cmd/restic/cmd_rewrite_integration_test.go:
add several new tests around the 'include' functionality

internal/filter/include.go:
this is where is include filter is defined

internal/walker/rewriter.go:
- struct RewriteOpts gains field 'KeepEmtpyDirectory', which is a 'NodeKeepEmptyDirectoryFunc()'
  which defaults to nil, so that al subdirectories are kept
- function 'NewSnapshotSizeRewriter()' gains the parameter 'keepEmptyDirecoryFilter' which
  controls the management of empty subdirectories in case of include filters active

internal/data/tree.go:
gains a function Count() for checking the number if node elements in a newly built tree

internal/walker/rewriter_test.go:
function 'NewSnapshotSizeRewriter()' gets an additional parameter nil to keeps things happy

cmd/restic/cmd_repair_snapshots.go:
function 'filterAndReplaceSnapshot()' gets an additional parameter 'keepEmptySnapshot=nil'

doc/045_working_with_repos.rst:
gets to mention include filters

changelog/unreleased/issue-4278:
the usual announcement file

git rebase master -i produced this

restic rewrite include - keep linter happy

cmd/restic/cmd_rewrite_integration_test.go:
linter likes strings.Contain() better than my strings.Index() >= 0
2026-01-31 19:42:56 +00:00
Michael Eischer
083cdf0675
Merge pull request #5613 from MichaelEischer/tree-node-iterator 2026-01-31 20:10:57 +01:00
Michael Eischer
ce7c144aac data: add support for unknown keys to treeIterator
While not planned, it's also not completely impossible that a tree node
might get additional top-level fields. As the tree iterator is built
with a strict expectation of the top-level fields, this would result in
a parsing error. Future-proof the code by simply skipping unknown
fields.
2026-01-31 20:03:38 +01:00
Michael Eischer
81948937ca data: test DualTreeIterator 2026-01-31 20:03:38 +01:00
Michael Eischer
fa8889eec4 data: test LoadTree+SaveTree cycle 2026-01-31 20:03:38 +01:00
Michael Eischer
6de64911fb data: test TreeFinder 2026-01-31 20:03:38 +01:00
Michael Eischer
17688c2313 data: move TestTreeMap to data package to allow reuse 2026-01-31 20:03:38 +01:00
Michael Eischer
e1a5550a27 test: use generics in Equal function signature
This simplifies comparing a typed value against nil. Previously it was
necessary to case nil into the proper type.
2026-01-31 20:03:38 +01:00
Michael Eischer
24d56fe2a6 diff: switch to efficient DualTreeIterator
The previous implementation stored the whole tree in a map and used it
for checking overlap between trees. This is now replaced with the
DualTreeIterator, which iterates over two trees in parallel and returns
the merge stream in order. In case of overlap between both trees, it
returns both nodes at the same time. Otherwise, only a single node is
returned.
2026-01-31 20:03:38 +01:00
Michael Eischer
350f29d921 data: replace Tree with TreeNodeIterator
The TreeNodeIterator decodes nodes while iterating over a tree blob.
This should reduce peak memory usage as now only the serialized tree
blob and a single node have to alive at the same time. Using the
iterator has implications for the error handling however. Now it is
necessary that all loops that iterate through a tree check for errors
before using the node returned by the iterator.

The other change is that it is no longer possible to iterate over a tree
multiple times. Instead it must be loaded a second time. This only
affects the tree rewriting code.
2026-01-31 20:03:38 +01:00
Michael Eischer
1e183509d4 data: rework StreamTrees to use synchronous callbacks
The tree.Nodes will be replaced by an iterator to loads and serializes
tree node ondemand. Thus, the processing moves from StreamTrees into the
callback. Schedule them onto the workers used by StreamTrees for proper
load distribution.
2026-01-31 20:03:38 +01:00
Michael Eischer
25a5aa3520 dump: fix missing error handling if tree cannot be read 2026-01-31 19:18:36 +01:00
Michael Eischer
278e457e1f data: use data.TreeWriter to serialize&write data.Tree
Always serialize trees via TreeJSONBuilder. Add a wrapper called
TreeWriter which combines serialization and saving the tree blob in the
repository. In the future, TreeJSONBuilder will have to upload tree
chunks while the tree is still serialized. This will a wrapper like
TreeWriter, so add it right now already.

The archiver.treeSaver still directly uses the TreeJSONBuilder as it
requires special handling.
2026-01-31 19:18:36 +01:00
Michael Eischer
f84d398989 repository: prevent test deadlock within WithBlobUploader
Calling t.Fatal internally triggers runtime.Goexit . This kills the
current goroutine while only running deferred code. Add an extra context
that gets canceled if the go routine exits while within the user
provided callback.
2026-01-31 19:18:36 +01:00
Michael Eischer
d82ea53735 data: fix invalid trees used in test cases
data.TestCreateSnapshot which is used in particular by TestFindUsedBlobs
and TestFindUsedBlobs could generate trees with duplicate file names.
This is invalid and going forward will result in an error.
2026-01-31 19:18:36 +01:00
Michael Eischer
34fdf5ba96
Merge pull request #5636 from MichaelEischer/clarify-parameter-docs 2026-01-31 19:15:24 +01:00
Michael Eischer
70591f00ed
Merge pull request #5690 from restic/backend-no-restic-imports 2026-01-31 19:13:07 +01:00
Michael Eischer
4bc6bb7e27 slightly reduce redundant wording 2026-01-31 19:07:06 +01:00
Michael Eischer
2628daba97 CI: prevent backends from importing internal/restic package 2026-01-31 12:00:04 +01:00
Martin Castillo
2269ec82e1 man: add note about append-only mode to restic-unlock.1
Since restic unlock _removes_ locks, a user might wonder
whether this works in append-only mode, which prevents deletion of
data.  This change explicitly mentions in the man page that the deletion
of locks is an allowed exception.
2026-01-27 12:07:47 +01:00
Winfried Plappert
86ccc6d445
Bugfix: restic check: add missing finalizeSnapshotFilter() (#5644)
Some checks failed
Create and publish a Docker image / build-and-push-image (push) Has been cancelled
test / Linux Go 1.24.x (push) Has been cancelled
test / Linux (race) Go 1.25.x (push) Has been cancelled
test / Windows Go 1.25.x (push) Has been cancelled
test / macOS Go 1.25.x (push) Has been cancelled
test / Linux Go 1.25.x (push) Has been cancelled
test / Cross Compile for subset 0/3 (push) Has been cancelled
test / Cross Compile for subset 1/3 (push) Has been cancelled
test / Cross Compile for subset 2/3 (push) Has been cancelled
test / lint (push) Has been cancelled
test / docker (push) Has been cancelled
Create and publish a Docker image / provenance (push) Has been cancelled
test / Analyze results (push) Has been cancelled
add missing finalizeSnapshotFilter() to cmd.RunE()

---------

Co-authored-by: Michael Eischer <michael.eischer@fau.de>
2026-01-26 21:08:15 +00:00
Michael Eischer
d0a5d0e2f7
Merge pull request #5657 from restic/dependabot/go_modules/github.com/spf13/cobra-1.10.2
build(deps): bump github.com/spf13/cobra from 1.10.1 to 1.10.2
2026-01-26 21:52:26 +01:00
Michael Eischer
fa13f1895f
Merge pull request #5658 from restic/dependabot/go_modules/github.com/elithrar/simple-scrypt-1.4.0
build(deps): bump github.com/elithrar/simple-scrypt from 1.3.0 to 1.4.0
2026-01-26 21:48:28 +01:00