before, borg extract always emitted the full file list at info log level.
now, you need to give --list to get the full file list (consistent with borg create --list).
this is currently only useful if you also use other output-generating options,
e.g. --show-rc - you can now leave away --list to only get that other output.
later, if extract gets more output-generating options, --list will get more useful.
every chunk has the encryption key type as first byte and we do not want to rewrite the whole repo
to change the passphrase type to repokey type. thus we simply dispatch this type to repokey
handler.
if there is a repokey that contains the same secrets as they were derived from the passphrase, it will just work.
if there is none yet, one needs to run migrate-to-repokey command to create it.
refactorings:
- introduced concept of default answer:
if the answer string is in the defaultish sequence, the return value of yes() will be the default.
e.g. if just pressing <enter> when asked on the console or if an empty string or "default" is
in the environment variable for overriding.
if an environment var has an invalid value and no retries are enabled: return default
if retries are enabled, next retry won't use the env var again, but either ask via input().
- simplify:
only one default - this should be a SAFE default as it is used in some special conditions
like EOF or invalid input with retries disallowed.
no isatty() magic, the "yes" shell command exists, so we could receive input even if it is not from a tty.
- clean:
separate retry flag from retry_msg
one of the biggest issues with borg < 1.0 was that it had a default target chunk
size of 64kiB, thus it created a lot of chunks, a huge chunk management overhead
(high RAM and disk usage).
this is available in python 3.4+.
note:
before removing the pbkdf tests, i ran them with the pbkdf from stdlib to make sure it gives same result.
long term testing of this now belongs into stdlib tests, not into borg.
we now use -P for --prefix and -p for --progress. previously, the
result of -p depended on the command: some were using it for
--progress, some for --prefix. this was confusing and was making it
impossible to both --progress and --prefix with on-letter options
--progress is likely used more often and interactively, so it get the
keystroke shortcut (lower "-p")
--prefix is used more rarely / in scripts, but important/dangerous for
prune, so it get the extra keystroke (higher "-P")
If somebody used -p someprefix and does not fix that to -P, it will
result in "no archive specified" or "unrecognized argument". So it
will neither cause pruning to remove wrong data nor go unnoticed.
Closes: #563
The fnmatch module in Python's standard library implements a pattern
format for paths which is similar to shell patterns. However, “*”
matches any character including path separators. This newly introduced
pattern syntax with the selector “sh” no longer matches the path
separator with “*”. Instead “**/” can be used to match zero or more
directory levels.
There are already three different styles and a fourth will be added.
A definition list is easier to navigate when trying to find the
description of a specific style.
This change implements the functionality requested in issue #361:
extracting files with a given extension. It does so by permitting
patterns to be used instead plain prefix paths. The pattern styles
supported are the same as for exclusions.
The “extract” command supports extracting all files underneath a given
set of prefix paths. The forthcoming support for extracting files using
a pattern (i.e. only files ending in “.zip”) requires the introduction
of path prefixes as a third pattern style, making it also available for
exclusions.
A function to parse pattern specifications was introduced in commit
2bafece. Since then it had a hardcoded default style of “fm”, meaning
fnmatch. With the forthcoming support for extracting files using
patterns this default style must be more flexible.
The utility functions “adjust_patterns” and “exclude_path” produce
respectively use a standard list object containing pattern objects.
With the forthcoming introduction of patterns for filtering files
to be extracted it's better to move the logic of these classes into
a single class.
The wrapper allows adding any number of patterns to an internal list
together with a value to be returned if a match function finds that
one of the patterns matches. A fallback value is returned otherwise.
- Stop using “adjust_pattern” and “exclude_path” as they're utility
functions not relevant to testing pattern classes
- Cover a few more cases, especially with more than one path separator
and relative paths
- At least one dedicated test function for each pattern style as opposed
to a single, big test mixing styles
- Use positive instead of negative matching (i.e. the expected list of
resulting items is a list of items matching a pattern)
