Commit graph

1088 commits

Author SHA1 Message Date
Łukasz Stelmach
c9c1403685 read a passphrase from a file descriptor
Read a passpharase from a file descriptor specified in the
BORG_PASSPHRASE_FD environment variable.
2018-11-05 17:29:56 +01:00
Thomas Waldmann
6754634ad7 acl platform code: fix acl set return type (1.0-maint) 2018-07-05 22:52:47 +02:00
Thomas Waldmann
cb5d2f38c2 xattr: add linux {list,get,set}xattr ctypes prototypes
(cherry picked from commit 71b02912c9)
2018-07-03 23:18:14 +02:00
Thomas Waldmann
b4e9397108 xattr: fix darwin flistxattr ctypes prototype
(cherry picked from commit 2d37c9d3f9)
2018-07-03 23:18:05 +02:00
Thomas Waldmann
69778e4c74 xattrs: fix borg exception handling on ENOSPC error, fixes #3808
(cherry picked from commit 959beb867b)
2018-05-18 17:24:17 +02:00
Thomas Waldmann
3d73f9849d remove platform.uname() call, fixes #3732
(cherry picked from commit 4f45eb660a)
2018-04-04 01:09:43 +02:00
Thomas Waldmann
b55767356c run platform.* calls within a cleaned env, fixes #3732
also: add exception handler around deprecated platform.linux_distribution() call.
(cherry picked from commit 1e94211bf5)
2018-04-04 01:08:28 +02:00
Thomas Waldmann
5d1ef09a5e add parens for C preprocessor macro argument usages
this is needed for correctness because the preprocessor is just
doing text replacement.

This is the correct way:

#define MUL(x, y) ((x) * (y))

MUL(1+2, 3-4) -> ((1+2) * (3-4))    # not: (1+2 * 3-4)

I didn't put parens around all arg usages for readability.

Some stuff (like index) is not expected to be an expression.

Also, when a arg is only used in another macro or function call,
no parens are needed either.

I reviewed the code: no harm was done (yet) due to this fault.

Thanks to @rciorba who found this.

(cherry picked from commit a3cecf599f)
(cherry picked from commit 981a936f47)
2017-12-23 18:56:46 +01:00
Thomas Waldmann
dab45ced20 set bsdflags last (include immutable flag), fixes #3263
(this was discovered in borg 1.1 for Linux, but applies to OSX/BSD as well)
2017-12-09 03:06:00 +01:00
Milkey Mouse
f1164b131b Clarify encrypted key format for borg key export (fixes #3296)
(cherry picked from commit 68bb3792fb)
2017-12-09 01:51:29 +01:00
Thomas Waldmann
213d8e2051 show excluded dir with "x" for tagged dirs / caches, fixes #3189
(cherry picked from commit 75004e9065)
2017-10-21 00:01:22 +02:00
Thomas Waldmann
936fce91c6 update docs about hardlinked symlinks limitation
(cherry picked from commit d535c6a25d)
2017-10-17 22:04:33 +02:00
Thomas Waldmann
a0c576d7b9 fix detection of non-local path, fixes #3108
filenames like ..foobar are valid, so, to detect stuff in upper dirs,
we need to include the path separator and check if it starts with '../'.

(cherry picked from commit 60e9249100)
2017-10-11 04:11:47 +02:00
Thomas Waldmann
5a5da99b51 fix LDLP restoration for subprocesses, fixes #3077
(cherry picked from commit b5069770b7)
(cherry picked from commit 7e4d6cac0f)
2017-10-01 22:25:29 +02:00
TW
f8246ce6bc Merge pull request #3073 from ThomasWaldmann/fix-partial-lines-1.0
remote: deal with partial lines (1.0-maint backport)
2017-09-30 03:33:53 +02:00
Thomas Waldmann
a5bfeddffe remote: deal with partial lines, fixes #2637
due to block buffering (in borg, pipes, sshd, ssh) partial lines might
be received. for plain text, this causes cosmetic issues.

the code now makes sure handle_remote_line() only gets called with a
complete line (which is terminated by any universal newline char, a
pure \r seems to be needed for remote progress displays).

it also fixes a yet undiscovered partial utf-8-sequence decoding issue
that might occur for the same reason.

(cherry picked from commit f2b48cd5c8)
2017-09-30 02:41:15 +02:00
Thomas Waldmann
315342edd2 use prepared env for xattr module's fakeroot version check
(cherry picked from commit 37a5442bef)
2017-09-30 02:00:24 +02:00
Thomas Waldmann
1ecab600f8 use prepared env for borg umount
(cherry picked from commit d6f810560c)
2017-09-30 01:54:12 +02:00
Thomas Waldmann
d5d2d6419f use prepared env for calling BORG_PASSCOMMAND, fixes #3050
(cherry picked from commit 2f7d7bdbb8)
2017-09-30 01:47:05 +02:00
Thomas Waldmann
55702777bb refactor/fix subprocess env preparation
refactor: make a generally usable function

fix: remove support code for ancient pyinstaller

the "else" branch was needed for pyinstaller < 20160820 because it did
not have the LD_LIBRARY_PATH_ORIG env var, so we just killed LDLP
because we had no better way.

but with borg tests running under fakeroot, this is troublesome as
fakeroot uses this also and can't find its library without it.

so, just remove it, we do not need to support old pyinstaller.

(cherry picked from commit 5dd16672c0)
2017-09-30 01:28:43 +02:00
Thomas Waldmann
5007e3fbb4 docs: change-passphrase only changes the passphrase, fixes #2990
(cherry picked from commit b6abee4d68)
2017-09-23 22:17:18 +02:00
Thomas Waldmann
9529895ed4 get rid of datetime.isoformat to avoid bugs like #2994
(cherry picked from commit da2f8dbe81)
2017-09-10 05:58:03 +02:00
Thomas Waldmann
9c3bae46e0 use safe parse_timestamp to parse timestamps, fixes #2994
also: refactor so it is possible to get tz-unaware datetime
objects from parse_timestamp.

(cherry picked from commit a9aa3c5f34)
2017-09-10 05:41:08 +02:00
Thomas Waldmann
da30e54fc4 use ISO_FORMAT* constants
(cherry picked from commit 457f5ceb30)
2017-09-10 05:27:59 +02:00
Thomas Waldmann
909c68a0d7 add ISO_FORMAT* constants to helpers module
no constants module here, so put it into helpers.
2017-09-10 05:17:40 +02:00
Thomas Waldmann
2798af3232 repo cleanup/write: invalidate cached FDs
(cherry picked from commit 88e1340445)
2017-09-07 16:49:32 +02:00
Thomas Waldmann
90fdfd26d8 repository: test FD invalidation 2017-09-06 15:17:05 +02:00
Marian Beermann
aa30be296b keymanager: don't depend on optional readline module
(cherry picked from commit b8793d9577)
2017-08-30 11:29:32 +02:00
TW
faa681ed06 Merge pull request #2971 from ThomasWaldmann/fix-timestamp-option-1.0
borg create --timestamp: set start time, fixes #2957
2017-08-27 20:46:06 +02:00
Marian Beermann
3b53823cbe create: document exclusion through nodump
(cherry picked from commit 495f838d88)
2017-08-27 15:42:45 +02:00
Thomas Waldmann
1d52b592cf borg create --timestamp: set start time, fixes #2957
(cherry picked from commit ac404e3a74)
2017-08-27 15:26:32 +02:00
enkore
1163cb5179 Buffer: fix wrong thread-local storage use (#2951)
Buffer: delete support for multiple threads

Thread-local objects are a fairly complex footgun, so avoid them in
the first place. When a Compressor uses a Buffer (for example),
just create multiple Compressor instances for each thread, owned by
each thread. Minimize multiple ownership of objects across
threads if at all possible!

(cherry picked from commit f7ed5d7220)
2017-08-25 05:12:52 +02:00
Thomas Waldmann
5beaa5bd02 ignore corrupt files cache, fixes #2939
ignore the files cache when corrupt and emit a warning message
so the users notices that there is a problem.
2017-08-19 01:03:38 +02:00
Thomas Waldmann
230e0feac6 migrate locks to child PID when daemonize is used
(cherry picked from commit 09e3a02fbc)
2017-08-14 00:25:51 +02:00
Thomas Waldmann
3a7f0fea69 fix exitcode of borg serve, fixes #2910
(cherry picked from commit 583de3eeb1)
2017-08-06 17:38:04 +02:00
Radu Ciorba
2cdd4353b6 replace modulo with if to check for wraparound in hashmap
Integer division is slow, and this improves the speed of all operations on the hashmap.

Benchmarked this patch on the rciorba/master-bench branch:
9e5d61e03c/results.html

(cherry picked from commit 12e0f55991)
2017-08-01 06:01:03 +02:00
TW
81b1442268 Merge pull request #2843 from ThomasWaldmann/checkpoint-interval
change --checkpoint-interval default from 600s to 1800s, fixes #2841
2017-07-19 15:54:36 +02:00
Thomas Waldmann
f25e3177e9 change --checkpoint-interval default from 600s to 1800s, fixes #2841
a user was (rightfully) complaining that borg does a whole lot of
writing to .cache and was also quite inefficient because that took
rather long and occurred too frequently.

this was caused by big indexes/caches and the relatively high default
checkpoint frequency (every 300s) in 1.0.

we already have increased that value to 1800s in master (which soon will
be 1.1), so this changeset just does the same for 1.0.x.
2017-07-19 01:08:08 +02:00
Marian Beermann
a93f636151 archiver: add test for paperkey import, fix bug
(cherry picked from commit 78f0e5d473)
2017-07-16 12:36:04 -07:00
TW
d0203a237d use limited unpacker for outer key, fixes #2174 (#2790)
* use limited unpacker for outer key, fixes #2174

* fixup: higher inner key data size limit

(cherry picked from commit 9ca666907b)
2017-07-16 12:36:03 -07:00
Marian Beermann
6990beb69e xattr: test split_lstring
(cherry picked from commit 6c67b64ab6)
2017-07-16 12:36:03 -07:00
Marian Beermann
cf0d28a24f BORG_PASSCOMMAND: use same cmd-string splitting as BORG_RSH
(cherry picked from commit 4e6a771ee7)
2017-06-26 04:11:15 +02:00
TuXicc
8c1da1adda Added BORG_PASSCOMMAND environment variable (#2573)
(cherry picked from commit 578b76af3a)
2017-06-26 04:08:42 +02:00
Thomas Waldmann
f7b16112dc move get_limited_unpacker to helpers
also: move some constants to borg.constants
(cherry picked from commit 89f3cab6cd)
2017-06-26 03:12:12 +02:00
Thomas Waldmann
35b0f1f4f9 Manifest: use limited unpacker
(cherry picked from commit 6c2c51939d)
2017-06-26 02:25:20 +02:00
Marian Beermann
10b196d532 document pattern denial of service
(cherry picked from commit 1f5ddb6572)
2017-06-26 01:56:14 +02:00
edgewood
932fb9ec7a Split up parsing and filtering for --keep-within (#2726)
Split up parsing and filtering for --keep-within

Fixes #2610

Parse --keep-within argument early, via new method within_range passed
to argparse type=, so that better error messages can be given.

Also swallows ValueError stacktrace per the comment in the old code that
including it wasn't desirable.
2017-06-25 23:31:04 +02:00
TW
04aa426334 Merge pull request #2722 from ThomasWaldmann/backports5
even more backports
2017-06-23 16:24:33 +02:00
Thomas Waldmann
8deb4dfdda remove skipping the noatime tests on GNU/Hurd, fixes #2710
I recently installed GNU/Hurd 2017 and found the atime test works now.

(cherry picked from commit 49411d1c6c)
2017-06-22 01:03:32 +02:00
Marian Beermann
12b2e1fdd0 chunker: fix invalid use of types
With the argument specified as unsigned char *, Cython emits
code in the Python wrapper to convert string-like objects to
unsigned char* (essentially PyBytes_AS_STRING).

Because the len(data) call is performed on a cdef'd string-ish type,
Cython emits a strlen() call, on the result of PyBytes_AS_STRING.

This is not correct, since embedded null bytes are entirely possible.

Incidentally, the code generated by Cython was also not correct,
since the Clang Static Analyzer found a path of execution where
passing arguments in a weird way from Python resulted in strlen(NULL).

Formulated like this, Cython emits essentially:

c_buzhash(
 PyBytes_AS_STRING(data),
 PyObject_Length(data),
 ...
)

which is correct.

(cherry picked from commit faf2d0b537)
2017-06-21 15:31:33 +02:00