- Archives now have a new metadata field 'comment'
- 'info' command shows a comment if it's present
- 'create' command now has option '--comment' for adding comments to archives.
- A new command 'comment' is added for modifying the comments on existing
archives.
Resolves#842.
(Remote)Repository.close() is not a public API anymore, but a private
API. It shall not be used from within other classes than Repository
or it's tests. The proper way is to use a context manager now. However,
for RPC/Remote compatibility with Borg 1.0 it is kept and unchanged.
Repositories are not opened by __init__ now anymore, it is done
by binding it to a context manager. (This SHOULD be compatible both ways
with remote, since opening the repo is handled by a RepositoryServer method)
Decorators @with_repository() and @with_archive simplify
context manager handling and remove unnecessary indentation.
Backported to 1.0-maint
[1]
This worked incidentally because OSes tend to return at least one page
worth of data when EOF is not reached. Increasing WINDOW_SIZE beyond
the page size might have lead to data loss.
[2]
If read() of the passed Python object returned something not-bytes,
PyBytes_Size returns -1 (ssize_t) which becomes a very larger number for
memcpy()s size_t.
Previously, on 'borg diff', the output always had first the modifications, then
additions, and finally removals. Output may be easier to follow if the various
kinds of changes are interleaved. This commit is a simple solution that first
collects the output lines and sorts them by file path before printing. This new
behavior is optional and disabled by default. It can be enabled with '--sort'
command line option.
This option will be especially useful after the planned multi-threading changes
arrive. Multi-threading may shuffle the archive order of files making diff
output hard to follow without sorting.
Resolves#797.
New repository config item, repository.append_only, causes Borg to never
delete or append to existing data files. Hints and indices are handled
as before, old ones are deleted, because they can get quite large, but
are automatically reconstructed: no need to keep them.
When append_only is activated a file /path/to/repo/transactions/<NUMBER>
will be created for every commit.
Deleting all segments <NUMBER+1> and higher will rollback to that commit.
Note that this only influences Borg behaviour. Since repository config
can't be altered remotely (except for repository.key) this can't be
disabled when accessed remotely over SSH with "borg serve" as the
forced command.
This is only a feature to support the use case, and does not replace
appropriate file system permissions or monitoring.
Resolves#809
Makes it easy to see paths excluded by --exclude* options for testing of
regexes, and for ongoing monitoring that files desired for backup aren't
getting excluded accidentally.
AESKeyBase.decrypt makes less copies of the data, specifically data[41:],
which copies the payload, is gone. This is possible since this commit
makes our lil' crypto API compatible with simple buffers.
In Cython there is a syntax for creating memoryviews from buffer-enabled
objects, however, it doesn't support read-only buffers. ro_buffer gets
this job done, though, and also does the proper type checking
(by PyBUF_SIMPLE).
Note: msgpack doesn't support memoryviews. Otherwise we could avoid
copying the en/de-crypted output with the final "return out[:ctl]"
This commit leads to a solid 10-15 % increase in performance of hash-lists
and extract.
The hmac_sha256() function releases the GIL.
The main design goals of the new format:
- One file takes exactly one line of output
- The format is easy to read with typical, long list of changes
- Metadata changes are visible and easy to examine
- Unuseful information is not shown
Resolves#757.
(Remote)Repository.close() is not a public API anymore, but a private
API. It shall not be used from within other classes than Repository
or it's tests. The proper way is to use a context manager now. However,
for RPC/Remote compatibility with Borg 1.0 it is kept and unchanged.
Repositories are not opened by __init__ now anymore, it is done
by binding it to a context manager. (This SHOULD be compatible both ways
with remote, since opening the repo is handled by a RepositoryServer method)
Decorators @with_repository() and @with_archive simplify
context manager handling and remove unnecessary indentation.
- This is compatible except for {formatkeys}, which has been replaced
by "borg list --help"
- --list-format is deprecated, use --format instead
(using deprecated arguments will print a warning and an exit code of 1)
- borg list now supports the usual [PATH [PATHS…]] syntax and excludes
- Additional keys: csize, num_chunks, unique_chunks, NUL
- Supports guaranteed_available hashlib hashes
(to avoid varying functionality depending on environment)
(also, the other hashes are really obscure, like MD-4)
every chunk has the encryption key type as first byte and we do not want to rewrite the whole repo
to change the passphrase type to repokey type. thus we simply dispatch this type to repokey
handler.
if there is a repokey that contains the same secrets as they were derived from the passphrase, it will just work.
if there is none yet, one needs to run migrate-to-repokey command to create it.
refactorings:
- introduced concept of default answer:
if the answer string is in the defaultish sequence, the return value of yes() will be the default.
e.g. if just pressing <enter> when asked on the console or if an empty string or "default" is
in the environment variable for overriding.
if an environment var has an invalid value and no retries are enabled: return default
if retries are enabled, next retry won't use the env var again, but either ask via input().
- simplify:
only one default - this should be a SAFE default as it is used in some special conditions
like EOF or invalid input with retries disallowed.
no isatty() magic, the "yes" shell command exists, so we could receive input even if it is not from a tty.
- clean:
separate retry flag from retry_msg
this is available in python 3.4+.
note:
before removing the pbkdf tests, i ran them with the pbkdf from stdlib to make sure it gives same result.
long term testing of this now belongs into stdlib tests, not into borg.
The fnmatch module in Python's standard library implements a pattern
format for paths which is similar to shell patterns. However, “*”
matches any character including path separators. This newly introduced
pattern syntax with the selector “sh” no longer matches the path
separator with “*”. Instead “**/” can be used to match zero or more
directory levels.
This change implements the functionality requested in issue #361:
extracting files with a given extension. It does so by permitting
patterns to be used instead plain prefix paths. The pattern styles
supported are the same as for exclusions.
The “extract” command supports extracting all files underneath a given
set of prefix paths. The forthcoming support for extracting files using
a pattern (i.e. only files ending in “.zip”) requires the introduction
of path prefixes as a third pattern style, making it also available for
exclusions.
The utility functions “adjust_patterns” and “exclude_path” produce
respectively use a standard list object containing pattern objects.
With the forthcoming introduction of patterns for filtering files
to be extracted it's better to move the logic of these classes into
a single class.
The wrapper allows adding any number of patterns to an internal list
together with a value to be returned if a match function finds that
one of the patterns matches. A fallback value is returned otherwise.
