Commit graph

1307 commits

Author SHA1 Message Date
Thomas Waldmann
cd4f6b41ca create: only run stat_simple_attrs() once
the second call was done in stat_attrs().

this increases backup with lots of unchanged files performance by ~ 5%.
2019-04-08 21:34:09 +02:00
Thalian
8b6184f1b3 #4473 – Better document borg check --max-duration (#4478)
better document borg check --max-duration, fixes #4473
2019-03-31 18:36:18 +02:00
TW
85b010af5d
Merge pull request #4482 from fantasya-pbem/feature/4471_Sorted-commands-help-in-steps
sorted commands help in multiple steps, fixes #4471
2019-03-30 19:54:50 +01:00
Thalian
1d74783a02 [TASK] #4471 – borg help should print <command> list in sorted order: Step 15
Finally moved borg help and added a section comment.
2019-03-30 07:11:42 +01:00
Thalian
1e2fee8c1a [TASK] #4471 – borg help should print <command> list in sorted order: Step 14
Moved borg recreate.
2019-03-30 07:11:42 +01:00
Thalian
00793cba01 [TASK] #4471 – borg help should print <command> list in sorted order: Step 13
Moved borg prune.
2019-03-30 07:11:42 +01:00
Thalian
b7af4b5f91 [TASK] #4471 – borg help should print <command> list in sorted order: Step 12
Moved borg init, key, list after info.
2019-03-30 07:11:42 +01:00
Thalian
2eebe13d01 [TASK] #4471 – borg help should print <command> list in sorted order: Step 11
Moved borg rename between mount and serve.
2019-03-30 07:11:42 +01:00
Thalian
9bee80ba12 [TASK] #4471 – borg help should print <command> list in sorted order: Step 10
Moved borg info.
2019-03-30 07:11:42 +01:00
Thalian
aecd2f7fcb [TASK] #4471 – borg help should print <command> list in sorted order: Step 9
Moved borg extract.
2019-03-30 07:11:42 +01:00
Thalian
947b47a11a [TASK] #4471 – borg help should print <command> list in sorted order: Step 8
Moved borg export-tar.
2019-03-30 07:11:42 +01:00
Thalian
76380e9ccb [TASK] #4471 – borg help should print <command> list in sorted order: Step 7
Moved borg diff.
2019-03-30 07:11:42 +01:00
Thalian
28172553db [TASK] #4471 – borg help should print <command> list in sorted order: Step 6
Moved borg debug & delete.
2019-03-30 07:11:42 +01:00
Thalian
e6b43ee5e9 [TASK] #4471 – borg help should print <command> list in sorted order: Step 5b
Moved borg create.
2019-03-30 07:11:28 +01:00
Thalian
74635f38d6 [TASK] #4471 – borg help should print <command> list in sorted order: Step 5a
Moved borg compact & config.
2019-03-30 07:04:23 +01:00
Thalian
55522b11fd [TASK] #4471 – borg help should print <command> list in sorted order: Step 4
Moved borg check after borg break-lock.
2019-03-29 18:44:44 +01:00
Thalian
737d17cc3f [TASK] #4471 – borg help should print <command> list in sorted order: Step 3
Moved borg benchmark and borg break-lock to the top.
2019-03-29 18:40:28 +01:00
Thalian
2e42543d3d [TASK] #4471 – borg help should print <command> list in sorted order: Step 2
Move subparsers definition out of "# borg mount" to the top so the mount section can be moved downwards.
2019-03-29 18:34:09 +01:00
Thalian
018176771e [TASK] #4471 – borg help should print <command> list in sorted order: Step 1
Added command section markers that will help when moving the sections later.
2019-03-29 17:38:45 +01:00
user062
68aff8b8c4 FIX #4373 (#4470)
--stats does not work with --dry-run, emit error msg and exit, fixes #4373
2019-03-25 13:44:11 +01:00
Thomas Waldmann
f541d6e689 display msgpack version as part of sysinfo (e.g. in tracebacks) 2019-03-22 14:18:55 +01:00
TW
676b5ffa9a
Merge pull request #4460 from ThomasWaldmann/fix-exception-rc
borg command shall terminate with rc 2 for ImportErrors, fixes #4424
2019-03-21 00:46:24 +01:00
Thomas Waldmann
97ca421fef borg command shall terminate with rc 2 for ImportErrors, fixes #4424
also: removed unused import.
2019-03-20 15:14:07 +01:00
Thomas Waldmann
7e7ea69e92 add comment about hashtable sizes, fixes #2830 2019-03-20 14:35:05 +01:00
Thomas Waldmann
d79d505d89 fuse xattr tests: remove freebsd from might-be-broken list 2019-03-18 22:21:25 +01:00
Thomas Waldmann
f66c102cc5 bump platform api version 2019-03-18 21:51:02 +01:00
Thomas Waldmann
0686237484 make freebsd xattr platform code api compatible with linux, fixes #3952
i.e. prefix the keys with the namespace, so it is ns.key like on
linux.

still only dealing with the "user" namespace, like before.
in the "system" namespaces there are ACLs (we deal with them via the acl
api, so no problem) and stuff from pnfsd (not sure what exactly).

this change is needed because FreeBSD's FUSE code expects the xattr
keys to be in that format.

it is also needed for cross-platform data exchange, so e.g. if one wants to:
- create archive on linux, extract on freebsd - with "user.xxx" xattrs.
- or vice versa.

archives made with older borg versions on freebsd will still extract correctly
on freebsd (not on linux though) even though they do not have the
namespace prefixes in the archived metadata (it will be interpreted in
same way as if they were prefixed by "user." as we do not support any
other namespace anyway).

they do not have "
2019-03-18 21:40:46 +01:00
TW
5070401a3d
Merge pull request #4437 from ThomasWaldmann/pkg-config
use pkg-config / refactor setup code
2019-03-15 22:33:59 +01:00
Thomas Waldmann
5173b9f407 add pkg-config support, fixes #1925
1. BORG_*_PREFIX is checked (avoids lib detection via pkg-config).
2. pkg-config is tried
3. fallback to bundled C code (or failure in case of OpenSSL)

also:
- simplified code again
- removed (c) headers, nothing left of original code
2019-03-13 18:26:47 +01:00
TW
f20e6d751f
Merge pull request #4434 from ThomasWaldmann/file-changed-warning
determine whether a file has changed while being backed up, fixes #1750
2019-03-12 13:43:11 +01:00
Thomas Waldmann
b3751b107d determine whether a file has changed while being backed up, fixes #1750 2019-03-11 22:55:27 +01:00
Thomas Waldmann
6ae5530507 lrucache: regularly remove old FDs, fixes #4427 2019-03-11 02:38:24 +01:00
TW
d493806e5c
incremental repo check (#4422)
incremental repo check, fixes #1657
2019-03-10 20:21:22 +01:00
TW
42ea069ddc
Merge pull request #4401 from joey4/fix4359
timestamp for borg delete --info added
2019-03-04 23:06:58 +01:00
Thomas Waldmann
7ad5290501 redo stale lock handling, fixes #3986
drop BORG_HOSTNAME_IS_UNIQUE (please use BORG_HOST_ID if needed)

borg now always assumes it has a unique hostid - either automatically
from fqdn plus uuid.getnode() or overridden via BORG_HOST_ID.
2019-03-04 21:07:05 +01:00
Thomas Waldmann
cd857b6d8f fix backing up FIFOs. fixes #4394
The O_NONBLOCK caused EAGAIN errors borg did not deal with, so we
better block and wait until read data is available from e.g. a fifo.

Fixed the --read-special test so it creates a FIFO, feeds it from a
thread and backs it up with borg. Then extracts and checks if correct
data has been backed up.

Also: renamed flags_follow to flags_special_follow (its only intended
use is on symlinks to special files with --read-special).
2019-03-02 19:14:47 +01:00
Björn Ketelaars
a87f38e35f crypto: LibreSSL has HMAC_CTX_free and *HMAC_CTX_new
While here improve indentation (use 4 spaces instead of 3).
2019-02-28 07:06:45 +01:00
TW
de151cd338
Merge pull request #4407 from bket/libressl
1.2.0a: fix LibreSSL support, see #4403
2019-02-26 22:31:49 +01:00
Thomas Waldmann
dbe92b7ace xattr.is_enabled: first listxattr before trying getxattr, see #4403
this is needed for the xattr dummy implementation (used e.g.
on openbsd):
- it returns [] for listxattr
- it fails with ENOATTR when trying getxattr for any name

also: reduce code duplication
2019-02-26 21:41:50 +01:00
Björn Ketelaars
9278920bd5 1.2.0a: fix LibreSSL support, see #4403 2019-02-26 21:24:28 +01:00
Björn Ketelaars
c5e3f3b4d4 acl_get() and acl_set() should accept a fd. Related to #4403
Fixes:

self = <borg.archive.MetadataCollector object at 0x14ffb70b28d0>
st = os.stat_result(st_mode=16895, st_ino=246007, st_dev=65280, st_nlink=3, st_uid=100
0, st_gid=0, st_size=512, st_atime=1551123978, st_mtime=1551123979, st_ctime=155112397
9)
path = 'input', fd = 16

    def stat_ext_attrs(self, st, path, fd=None):
        attrs = {}
        bsdflags = 0
        with backup_io('extended stat'):
            if not self.nobsdflags:
                bsdflags = get_flags(path, st, fd=fd)
            xattrs = xattr.get_all(fd or path, follow_symlinks=False)
>           acl_get(path, attrs, st, self.numeric_owner, fd=fd)
E           TypeError: acl_get() got an unexpected keyword argument 'fd'

src/borg/archive.py:1023: TypeError
2019-02-26 17:06:58 +01:00
Thomas Waldmann
db2b385a8b fix openat/statat issues for root directory, fixes #4405
basename('/') == '' and we can't use that for openat/statat.
also, there is no parent FD for the root directory...
2019-02-26 11:18:37 +01:00
Joey
29e7701c3b timestamp for borg delete --info added, fixes #4359 2019-02-25 12:20:02 +01:00
Thomas Waldmann
a65cefb7bb bump API_VERSIONs to 1.2_xx 2019-02-24 19:45:41 +01:00
TW
886cbdca8e
Merge pull request #4393 from donnyward/master
Update bundled zstd code to 1.3.8, fixes #4210
2019-02-24 15:15:57 +01:00
Thomas Waldmann
c832822721 remove x bits from files, no content changes 2019-02-24 14:15:49 +01:00
Donny Ward
1c3ace4724 Update bundled zstd code to 1.3.8, fixes #4210
Didn't add dll folder since it wasn't copied when updating zstd to
1.3.4.

Didn't add extra files from the lib directory (Makefile, README.md, etc)
that were also left out when updating zstd to 1.3.4.

Add new zstd files to setup_zstd.py (DDict was refactored in
facebook/zstd#1388)
2019-02-23 12:46:11 -08:00
Thomas Waldmann
6809f6f7fa calc_stats: use archive stats metadata, if available
by default, we still have to compute unique_csize the slow way,
but the code offers want_unique=False param to not compute it.
2019-02-23 15:05:07 +01:00
Thomas Waldmann
e569595974 include size/csize/nfiles[_parts] stats into archive, fixes #3241 2019-02-23 15:05:07 +01:00
Thomas Waldmann
58f177aa82 add comment about unused recreate_* members in ArchiveItem 2019-02-23 10:49:24 +01:00