Commit graph

5251 commits

Author SHA1 Message Date
Thomas Waldmann
be16b167ea rename test_mount_hardlinks to test_fuse_mount_hardlinks
so both can be excluded by "not test_fuse"

(fedora does this for testing)
2019-05-16 00:17:19 +02:00
TW
ddf615149a
Merge pull request #4560 from ThomasWaldmann/bundle-msgpack
Bundle msgpack
2019-05-15 21:16:26 +02:00
TW
0f6aa52106
Merge pull request #4564 from ThomasWaldmann/placeholders-in-comments
enable placeholder usage at some places
2019-05-14 21:35:03 +02:00
Thomas Waldmann
7f0e07603e enable placeholder usage in --comment, fixes #4559 2019-05-14 14:51:57 +02:00
Thomas Waldmann
fb05da42cf enable placeholder usage in --glob-archives, fixes #4495
they are supported for --prefix already, so it totally makes sense to
also support them for the more powerful/generic --glob-archives.
2019-05-14 14:31:37 +02:00
Thomas Waldmann
fee6b5ded4 add msgpack license to docs/3rd_party/msgpack 2019-05-14 00:23:13 +02:00
Thomas Waldmann
5fa05ffed9 move msgpack hints to helpers.py
also remove msgpack from install_requires - by default we use the
bundled code now.
2019-05-14 00:00:47 +02:00
Thomas Waldmann
4d1cacca66 use bundled msgpack (optionally use external msgpack)
based on a patch by felix schwarz, thanks!

changes after that:

import msgpack from borg.helpers (as master branch does)

there:
- try importing it from borg.algorithms.msgpack bundled code.
- if that does not work, try importing external msgpack.
- if it is desired to import the external msgpack without ripping
  out the bundled code, set prefer_system_msgpack = True.

fix msgpack.fallback reference, fix msgpack import.
2019-05-13 23:32:11 +02:00
Thomas Waldmann
f9cb04079d setup.py: also build a bundled msgpack
based on a patch by felix schwarz, thanks!

changes after that:

build msgpack cpp files at release time, do not use cython otherwise.

we want to include .cpp (and .c) files into the sdist, so users
installing the pypi package won't need to have cython installed.

also: python setup.py clean shall also cleanup the *.cpp and binaries
built from that.

.gitignore: ignore generated msgpack .cpp sources
setup.cfg: flake8: ignore 3rd party bundled msgpack source
2019-05-13 23:29:17 +02:00
Thomas Waldmann
8f271ed7fc git mv _msgpack algorithms/msgpack
we have all other bundled stuff there already.
2019-05-13 23:27:18 +02:00
Thomas Waldmann
abda1a4463 patch msgpack to use relative imports
patch by felix schwarz, thanks!
2019-05-13 22:49:03 +02:00
Thomas Waldmann
383807aed7 bundle msgpack 0.5.6 "as is" as borg._msgpack 2019-05-13 22:49:03 +02:00
TW
c78c3ff8a4
Merge pull request #4561 from ThomasWaldmann/filter-msgpack-warning
pure-py msgpack warning shall not make a lot of tests fail, fixes #4558
2019-05-13 18:21:36 +02:00
Thomas Waldmann
690195a9a3 pure-py msgpack warning shall not make a lot of tests fail, fixes #4558
if the tests use a pure-python msgpack, 1 test which is specifically
made for that, will fail. esp. for linux distribution packages, this
will still point to the problem (that the msgpack package is not built
or installed correctly).
2019-05-13 17:10:14 +02:00
TW
3e90eb1d05
Merge pull request #4487 from ThomasWaldmann/nofollow-check
do not check python/libc for borg serve, fixes #4483
2019-05-12 19:08:39 +02:00
TW
d34b6a53e2
Merge pull request #4555 from ThomasWaldmann/update-changes-1.1
Update changes (1.1)
2019-05-12 19:05:26 +02:00
Thomas Waldmann
aca9855f80 update CHANGES (1.1-maint)
for 1.1.10 mostly, also adding 1 item that was forgotton for 1.1.9.
2019-05-12 03:08:50 +02:00
Thomas Waldmann
e0735e5d4f changelog: add 1.1.10 section 2019-05-12 03:03:29 +02:00
TW
d07d51e35e
Merge pull request #4554 from ThomasWaldmann/whitelist-client-options-1.1
use whitelist approach for borg serve, fixes #4097 (1.1-maint backport)
2019-05-12 02:53:50 +02:00
TW
7010cda6f6
Merge pull request #4553 from ThomasWaldmann/coverage-warning-1.1
get rid of confusing coverage warning, fixes #2069 (1.1-maint backport)
2019-05-11 23:37:30 +02:00
Thomas Waldmann
185282776e use whitelist approach for borg serve, fixes #4097 2019-05-11 23:13:11 +02:00
TW
b033693c9a
Merge pull request #4542 from ThomasWaldmann/fix-extract-hardlinks-1.1
slave hardlinks extraction issue, see #4350 (1.1-maint)
2019-05-11 22:15:16 +02:00
Thomas Waldmann
f2d8e3a93a get rid of confusing coverage warning, fixes #2069 2019-05-11 21:35:37 +02:00
TW
8a42028613
Merge pull request #4552 from ThomasWaldmann/upgrade-openbsd-box
vagrant: upgrade openbsd box to 6.4
2019-05-11 21:28:16 +02:00
Thomas Waldmann
847cc52209 vagrant: upgrade openbsd box to 6.4
6.0 went out of support, the ftp server URL we used 404ed and we
already had the 6.4 box for master branch, so just use it for
1.1-maint also.
2019-05-11 21:27:13 +02:00
Thomas Waldmann
18242ab9e2 preload chunks for hardlink slaves w/o preloaded master, fixes #4350
also split the hardlink extraction test into 2 tests.

(cherry picked from commit f33f318d81)
2019-05-06 20:11:18 +02:00
TW
b30eb36333
Merge pull request #4530 from shawnl/forward-port
2 back ports
2019-05-01 23:47:05 +02:00
Thomas Waldmann
e886df4cde create: only run stat_simple_attrs() once
This is a backport of cd4f6b41ca

the second call was done in stat_attrs().

this increases backup with lots of unchanged files performance by ~ 5%.
2019-04-30 15:28:29 -05:00
user062
133da4dd49 FIX #4373 (Incorrect borg prune --stats output with --dry-run)
This is a backport of 68aff8b8c4
2019-04-27 09:05:08 -05:00
TW
3bb9efc784
Merge pull request #4503 from mikenye/1.1-maint
Add "SSH Configuration" section to "borg serve" (backport from master)
2019-04-09 11:04:04 +02:00
Mike Nye
016942a222 modified: serve.rst
Add "SSH Configuration" section to "borg serve" documentation, to outline ssh/sshd configuration to prevent borg serve keeping a lock on a repo in the event the ssh connection is abnormally disconnected.

In response to issues #3988, #636 and #4485 (and probably others).

Backported from master.
2019-04-09 08:57:38 +08:00
Thomas Waldmann
24118459c6 do not check python/libc for borg serve, fixes #4483
the check checks whether follow_symlinks=False is supported, which
requires that the glibc is recent enough / python was compiled
for a recent enough glibc. follow_symlinks=False is only used for borg
create and extract, but not needed for borg serve.

thus, this makes it possible to run "borg serve" even on a bit older
servers. be careful, due to the presence of this check on the server
side until now, older server systems are not really much tested.
2019-04-02 23:34:05 +02:00
TW
e281c8329c
Merge pull request #4475 from ThomasWaldmann/funding-link
README: new URL for funding options
2019-03-25 23:32:56 +01:00
Thomas Waldmann
d9c5dfb55b README: new URL for funding options 2019-03-25 23:15:39 +01:00
TW
c5132f2a2e
Merge pull request #4472 from ThomasWaldmann/freeze-test-env
lock travis-ci test env to py 3.4 compatible versions, fixes #4343
2019-03-25 00:00:51 +01:00
Thomas Waldmann
ff5bc4552f lock travis-ci test env to py 3.4 compatible versions, fixes #4343
maybe slightly newer will also work, e.g. pip < 19.2, pytest < 5.0.
2019-03-24 22:51:50 +01:00
Thomas Waldmann
867bf7d34f display msgpack version as part of sysinfo (e.g. in tracebacks)
1.1-maint backport
2019-03-22 14:24:35 +01:00
Thomas Waldmann
88b4a00d1b freebsd xattr namespace: str vs. bytes fixup 2019-03-21 03:20:12 +01:00
TW
3e2393a058
Merge pull request #4463 from ThomasWaldmann/fix-exception-rc-1.1
borg command shall terminate with rc 2 for ImportErrors, fixes #4424
2019-03-21 02:04:15 +01:00
TW
1cd8ca297a
Merge pull request #4458 from ThomasWaldmann/fix-4348-1.1
Fix 4348 (1.1-maint backport)
2019-03-21 01:49:16 +01:00
TW
8e78580c83
Merge pull request #4457 from ThomasWaldmann/freebsd-xattr-ns-1.1
make freebsd xattr platform code api compatible with linux, fixes #3952 (1.1-maint bp)
2019-03-21 01:48:49 +01:00
Thomas Waldmann
1cabd77029 borg command shall terminate with rc 2 for ImportErrors, fixes #4424
(cherry picked from commit 97ca421fef)
2019-03-21 01:27:13 +01:00
Milkey Mouse
96bd36d37a Add test for error upon exporting a key to a directory
(cherry picked from commit d26264bb70)
2019-03-20 14:22:39 +01:00
Milkey Mouse
891538ca7e Emit user-friendly error if repo key is exported to a directory (fixes #4348)
(cherry picked from commit 72ebd13e6f)
2019-03-20 14:22:27 +01:00
Thomas Waldmann
98198b34d9 make freebsd xattr platform code api compatible with linux, fixes #3952
i.e. prefix the keys with the namespace, so it is ns.key like on
linux.

still only dealing with the "user" namespace, like before.
in the "system" namespaces there are ACLs (we deal with them via the acl
api, so no problem) and stuff from pnfsd (not sure what exactly).

this change is needed because FreeBSD's FUSE code expects the xattr
keys to be in that format.

it is also needed for cross-platform data exchange, so e.g. if one wants to:
- create archive on linux, extract on freebsd - with "user.xxx" xattrs.
- or vice versa.

archives made with older borg versions on freebsd will still extract correctly
on freebsd (not on linux though) even though they do not have the
namespace prefixes in the archived metadata (it will be interpreted in
same way as if they were prefixed by "user." as we do not support any
other namespace anyway).

(cherry picked from commit 0686237484)
2019-03-20 14:13:59 +01:00
TW
0a20a03b7f
Merge pull request #4455 from ThomasWaldmann/backports-17
Backports to 1.1-maint (17)
2019-03-19 20:17:42 +01:00
Thomas Waldmann
0f72a8a0cf lrucache: regularly remove old FDs, fixes #4427 2019-03-19 18:52:19 +01:00
Thomas Waldmann
6ee42216bf avoid stale filehandle issues, fixes #3265 2019-03-19 18:51:23 +01:00
Thomas Waldmann
1efec3ced1 add comment about unused recreate_* members in ArchiveItem 2019-03-19 18:45:19 +01:00
Thomas Waldmann
8ad90c1242 remove ARCHIVE_KEYS duplication
also: get key set in sync, obviously we have "recreate_partial_chunks"
in ArchiveItem still.
2019-03-19 18:36:18 +01:00