Commit graph

1100 commits

Author SHA1 Message Date
Thomas Waldmann
e886df4cde create: only run stat_simple_attrs() once
This is a backport of cd4f6b41ca

the second call was done in stat_attrs().

this increases backup with lots of unchanged files performance by ~ 5%.
2019-04-30 15:28:29 -05:00
user062
133da4dd49 FIX #4373 (Incorrect borg prune --stats output with --dry-run)
This is a backport of 68aff8b8c4
2019-04-27 09:05:08 -05:00
Thomas Waldmann
867bf7d34f display msgpack version as part of sysinfo (e.g. in tracebacks)
1.1-maint backport
2019-03-22 14:24:35 +01:00
Thomas Waldmann
88b4a00d1b freebsd xattr namespace: str vs. bytes fixup 2019-03-21 03:20:12 +01:00
TW
3e2393a058
Merge pull request #4463 from ThomasWaldmann/fix-exception-rc-1.1
borg command shall terminate with rc 2 for ImportErrors, fixes #4424
2019-03-21 02:04:15 +01:00
TW
1cd8ca297a
Merge pull request #4458 from ThomasWaldmann/fix-4348-1.1
Fix 4348 (1.1-maint backport)
2019-03-21 01:49:16 +01:00
Thomas Waldmann
1cabd77029 borg command shall terminate with rc 2 for ImportErrors, fixes #4424
(cherry picked from commit 97ca421fef)
2019-03-21 01:27:13 +01:00
Milkey Mouse
96bd36d37a Add test for error upon exporting a key to a directory
(cherry picked from commit d26264bb70)
2019-03-20 14:22:39 +01:00
Milkey Mouse
891538ca7e Emit user-friendly error if repo key is exported to a directory (fixes #4348)
(cherry picked from commit 72ebd13e6f)
2019-03-20 14:22:27 +01:00
Thomas Waldmann
98198b34d9 make freebsd xattr platform code api compatible with linux, fixes #3952
i.e. prefix the keys with the namespace, so it is ns.key like on
linux.

still only dealing with the "user" namespace, like before.
in the "system" namespaces there are ACLs (we deal with them via the acl
api, so no problem) and stuff from pnfsd (not sure what exactly).

this change is needed because FreeBSD's FUSE code expects the xattr
keys to be in that format.

it is also needed for cross-platform data exchange, so e.g. if one wants to:
- create archive on linux, extract on freebsd - with "user.xxx" xattrs.
- or vice versa.

archives made with older borg versions on freebsd will still extract correctly
on freebsd (not on linux though) even though they do not have the
namespace prefixes in the archived metadata (it will be interpreted in
same way as if they were prefixed by "user." as we do not support any
other namespace anyway).

(cherry picked from commit 0686237484)
2019-03-20 14:13:59 +01:00
Thomas Waldmann
0f72a8a0cf lrucache: regularly remove old FDs, fixes #4427 2019-03-19 18:52:19 +01:00
Thomas Waldmann
6ee42216bf avoid stale filehandle issues, fixes #3265 2019-03-19 18:51:23 +01:00
Thomas Waldmann
1efec3ced1 add comment about unused recreate_* members in ArchiveItem 2019-03-19 18:45:19 +01:00
Thomas Waldmann
8ad90c1242 remove ARCHIVE_KEYS duplication
also: get key set in sync, obviously we have "recreate_partial_chunks"
in ArchiveItem still.
2019-03-19 18:36:18 +01:00
Joey
f904f473ff timestamp for borg delete --info added, fixes #4359 2019-03-05 11:28:16 +01:00
Thomas Waldmann
384173180b test_mount_hardlinks: get rid of fakeroot-caused test fails, fixes #3389 2019-02-11 00:09:10 +01:00
Thomas Waldmann
1b277cb1ff security fix: configure FUSE with "default_permissions", fixes #3903
"default_permissions" is now enforced by borg by default to let the
kernel check uid/gid/mode based permissions.

"ignore_permissions" can be given to not enforce "default_permissions".

note: man mount.fuse explicitly tells about the security issue:

    default_permissions
	By  default FUSE doesn't check file access permissions, ...
	This option enables permission checking, restricting access
	based on file mode.
	This option is usually useful together with the allow_other
	mount option.

We consider this a pitfall waiting for someone to fall into and this is
why we chose to change the default behaviour for borg.
2019-02-10 10:21:48 +01:00
TW
ca8040317d
Merge pull request #4326 from ThomasWaldmann/backports-16
1.1-maint backports (16)
2019-02-05 16:18:32 +01:00
Thomas Waldmann
b407b36320 fixup: compute all from sync.* 2019-02-05 14:42:55 +01:00
Thomas Waldmann
f8feaa7dd5 fixup: rename expect_chunks_map_key state to expect_map_key 2019-02-05 14:42:43 +01:00
Thomas Waldmann
e0480800d5 cache_sync: compute size/count stats, borg info: consider part files
fixes #3522
2019-02-05 14:42:31 +01:00
Thomas Waldmann
429645d174 borg init --make-parent-dirs parent1/parent2/repo_dir, fixes #4235 2019-02-05 14:32:56 +01:00
Thomas Waldmann
2378ef4890 add archive name to check --repair output, fixes #3447
so it does not look like duplicated and also informs the user about
affected archives.
2019-02-05 14:31:15 +01:00
TW
94851b0c5f
Merge pull request #4317 from ThomasWaldmann/workaround-wsl-sync_file_range-issue
work around some Microsoft WSL issues
2019-02-05 04:44:37 +01:00
TW
38b40e805e
Merge pull request #4318 from ThomasWaldmann/workaround-backslash-issue
work around the backslash issue, see #4280
2019-02-05 03:35:46 +01:00
Thomas Waldmann
59e067816e work around the backslash issue, see #4280
thanks to AJ Jordan @strugee for finding this.

while using \\ gives a single backslash when using "borg help pattern"
on the shell, our rendered html docs did not show the single backslash
at all.

i was lazy do dig into that issue, so i just used the word "backslash".

:-)
2019-02-05 01:56:42 +01:00
Thomas Waldmann
53649cd6e6 work around some Microsoft WSL issues
see there:

https://github.com/borgbackup/borg/issues/1961

and especially there (not implemented sync_file_range):

https://github.com/Microsoft/WSL/issues/645
2019-02-05 01:30:54 +01:00
Thomas Waldmann
cbe98e4f19 fix duplicate test method name, fixes #4311
thanks to Anthony Sottile for finding this!
2019-02-03 02:36:41 +01:00
Thomas Waldmann
7786eea531 check msgpack version, terminate for unsupported versions 2019-02-03 00:00:10 +01:00
TW
b995a1923d
Merge pull request #4298 from ThomasWaldmann/fix-short-hostname-1.1
make "hostname" short, even on misconfigured systems, fixes #4262
2019-02-01 14:22:45 +01:00
TW
8a523a15a3
Merge pull request #4299 from ThomasWaldmann/add-rsh-arg-1.1
Add --rsh command line option to complement BORG_RSH env var. Fixes #1701
2019-02-01 14:22:02 +01:00
Manuel Riel
e99448cc7b Add --rsh command line option to complement BORG_RSH env var. Fixes #… (#4290)
Add --rsh command line option to complement BORG_RSH env var. Fixes #1701
(cherry picked from commit 9ab0a095ab)
2019-02-01 11:21:50 +01:00
Thomas Waldmann
6614918e68 make "hostname" short, even on misconfigured systems, fixes #4262
(cherry picked from commit 111c34f92e)
2019-02-01 11:11:25 +01:00
Manu
aa73488e78 Use f_frsize instead of f_bsize to calculate free space. Fixes #4289
(cherry picked from commit c3a882b509)
2019-02-01 11:02:45 +01:00
Manu
38e6ff0a54 Quit with error message when no config key is provided. Fixes #4223 2019-01-31 22:34:54 +08:00
TW
e35091596d
Merge pull request #4273 from ThomasWaldmann/fix-empty-segment-crash
recover_segment: handle too small segment files correctly, see #4272
2019-01-29 19:18:21 +01:00
Thomas Waldmann
6a119cd410 correctly release memoryview, see #4243
keep "data" as is, use "d" for slices

so that the data.release() call is on the original memoryview and
also we can delete the last reference to a slice of it first.
2019-01-29 15:58:35 +01:00
Thomas Waldmann
c1ddabdef6 recover_segment: handle too small segment files correctly, see #4272
nothing left to recover there, but at least we must not crash in mmap().
2019-01-29 09:03:18 +01:00
Thomas Waldmann
bcdcc428a6 avoid diaper pattern in configparser by opening files, fixes #4263
this will fail early with correct error msg / exception traceback
if a config file is not readable.
2019-01-27 01:36:52 +01:00
Thomas Waldmann
c188a0b0bc add "# cython: language_level=3" directive to .pyx files, fixes #4214
in master branch, it was possible to do this via setup.py's
cythonize() calls, but we do not have them in 1.1.

thus, i modified the source files, setting the language level to 3,
which is supported since long in Cython (3str is only supported since
recently so might fail with older Cython versions).

Before this changeset, current Cython 0.29.x emitted a FutureWarning,
but still compiled. The next major release of Cython is said to emit
an error instead, so this changeset hopefully avoids the build breaking
after a Cython upgrade.
2019-01-05 20:12:32 +01:00
Thomas Waldmann
ca1928c925 on linux, symlinks can't have ACLs, see #4044 2018-12-08 23:11:08 +01:00
Thomas Waldmann
14464161ed invalid locations: give err msg containing parsed location, fixes #4179
(cherry picked from commit 65d8091245)
2018-11-28 01:56:58 +01:00
Thomas Waldmann
8859af731d list repo: add placeholders for hostname and username, fixes #4130
also: use a more generic get_meta() access method to avoid duplication.
(cherry picked from commit e9ac9beef0)
2018-11-28 00:26:26 +01:00
Thomas Waldmann
60347a4dd6 xattr key names must start with user. on linux
... or the call will just error with "operation not supported"

(cherry picked from commit 53971e8e79)
2018-11-17 01:30:57 +01:00
Thomas Waldmann
216d1f810f pytest4 complains if we directly call the fixture function, fixes #4172
just added a normal function with the code, call it from fixture and
also from unittest test method.

(cherry picked from commit d0a01c532e)
2018-11-16 21:56:38 +01:00
ashmodei
29edb0cf37 Fixed wrong an archive's name pattern in CRUD benchmark help. 2018-11-09 15:02:38 +03:00
ashmodei
81f10266e3 Added PATH::archive output for creation function on info log level. 2018-11-09 10:39:03 +03:00
Thomas Waldmann
4f1f98eee3 fixup: the help text is parsed as rst markup, so a lone * is an error
* and ** are interpreted as emphasis and need to be matched for begin
and end of emphasis. the \ in front of it is to get a non-markup *.

(cherry picked from commit 64a53ba43a)
2018-11-04 20:01:18 +01:00
Thomas Waldmann
76185efb84 flake8: fix W605 (invalid escape sequences)
(cherry picked from commit 5c66a60f79)
2018-11-04 19:58:46 +01:00
Thomas Waldmann
78c6d21a08 flake8: fix F841
(cherry picked from commit 10cdadb2f8)
2018-11-04 19:58:17 +01:00