Commit graph

1185 commits

Author SHA1 Message Date
Emmo Emminghaus
733a2bfa30 Introduce borg.platformflags.is_<os> 2018-11-10 23:34:43 +01:00
Emmo Emminghaus
973921df5a remove unneded code and remarks, added needed except clause 2018-11-10 21:56:39 +01:00
Emmo Emminghaus
558ca61d20 remove posix issues and fixup for unsupported methodes 2018-11-10 21:48:46 +01:00
Emmo Emminghaus
b997d5ba5b move code from borg.helpers.usergroup to borg.platform.posix 2018-11-10 21:43:45 +01:00
motwok
78a3f2475e Basic MSC Compatibility (#4147)
MSC Compatibility (Native Compile on Windows)
2018-11-03 18:52:54 +01:00
TW
08a03126b7
Merge pull request #4127 from motwok/master
c code portability fixes: issue #2677 and 2 more
2018-11-02 20:13:20 +01:00
Emmo Emminghaus
b2c3bc4d9e removing accidental change 2018-11-02 14:10:23 +01:00
step21
19c269cdcf Fixed link to cachedir spec in fs helper to address #4140 2018-10-31 14:05:49 +01:00
step21
0c99aee724 Fixed link to cachedir spec in docs to address #4140 2018-10-31 13:57:08 +01:00
step21
0d26ad3661 Explicitly convert environment variable to str to address #4136 2018-10-31 09:41:02 +01:00
step21
47490c79be Moving the flags of the compiled regex to the front to fix #4137 2018-10-31 07:56:16 +01:00
motwok
6fde90b7f8
Merge pull request #1 from borgbackup/master
Update form upstream to fork
2018-10-29 21:39:01 +01:00
Thomas Waldmann
64a53ba43a fixup: the help text is parsed as rst markup, so a lone * is an error
* and ** are interpreted as emphasis and need to be matched for begin
and end of emphasis. the \ in front of it is to get a non-markup *.
2018-10-29 13:53:40 +01:00
Thomas Waldmann
5c66a60f79 flake8: fix W605 (invalid escape sequences) 2018-10-29 12:36:07 +01:00
Thomas Waldmann
10cdadb2f8 flake8: fix F841 2018-10-29 12:36:03 +01:00
Emmo Emminghaus
718abfd6f8 cleanup void * in _hashindex.c (2nd) 2018-10-27 17:45:10 +02:00
Emmo Emminghaus
9d1276af04 cleanup void * in _hashindex.c 2018-10-27 17:34:25 +02:00
Emmo Emminghaus
f8ef6af454 hashindex: clean void* arithmetic up #2677
lowlevel: clean void* arithmetic up
unpack: repalce nonstandard false with 0
2018-10-24 21:40:05 +02:00
Manu
93b968cc35 Add argument storage_quota to forced_result. Avoids override by client. 2018-10-02 17:28:29 +08:00
Łukasz Stelmach
548355125e read a passphrase from a file descriptor
Read a passpharase from a file descriptor specified in the
BORG_PASSPHRASE_FD environment variable.
2018-09-21 09:11:39 +02:00
Henry-Joseph Audéoud
b66533170a Forward the format_spec to datetime class
If specified, the argument `format_spec` is forwarded from the method
`OutputTimestamp.__format__` to the method `format_time` method.  Thus,
we are able to specify the format, in a way supported by the class
`datetime`.

The default behaviour of `format_time` is not affected.  The other parts
of code that does not provide a `format_spec` are not affected by the
change.
2018-08-29 14:00:57 +02:00
TW
708a65a0b5
Merge pull request #4022 from ThomasWaldmann/finding-pieces-master
some more debug commands (master forward port)
2018-08-09 09:43:57 +02:00
Thomas Waldmann
eebc33a281 dump-repo-objs: filename layout improvements
(cherry picked from commit 24812b0317)
2018-08-09 08:29:46 +02:00
Thomas Waldmann
d6cb39a6d6 implement borg debug dump-repo-objs --ghost
intended as a last resort measure to export all segment file contents
in a relatively easy to use format.

if you want to dig into a damaged repo (e.g. missing segment files,
missing commits) and you know what you do.

note: dump-repo-objs --ghost must not use repo.list()

because this would need the repo index and call get_transaction_id and
check_transaction methods, which can easily fail on a damaged repo.

thus we use the same low level scan method as we use anyway to get
some encrypted piece of data to setup the decryption "key".

(cherry picked from commit 8738e85967)
2018-08-09 08:29:34 +02:00
Thomas Waldmann
a3a15ddf57 use repository.scan() for dump-repo-objs to get on-disk order
it is also more efficient, avoids random access.

(cherry picked from commit 2b7d65b024)
2018-08-09 08:29:24 +02:00
Thomas Waldmann
9445b00408 implement borg debug search-repo-objs searchterm
(cherry picked from commit c2ed0d9ee0)
2018-08-09 08:29:11 +02:00
Thomas Waldmann
4b31ad123f init: add warning to store both key and passphrase at safe place(s) 2018-08-09 07:25:49 +02:00
Thomas Waldmann
3c173cc03b wrap msgpack, fixes #3632, fixes #2738
wrap msgpack to avoid future upstream api changes making troubles
or that we would have to globally spoil our code with extra params.

make sure the packing is always with use_bin_type=False,
thus generating "old" msgpack format (as borg always did) from
bytes objects.

make sure the unpacking is always with raw=True,
thus generating bytes objects.

note:

safe unicode encoding/decoding for some kinds of data types is done in Item
class (see item.pyx), so it is enough if we care for bytes objects on the
msgpack level.

also wrap exception handling, so borg code can catch msgpack specific
exceptions even if the upstream msgpack code raises way too generic
exceptions typed Exception, TypeError or ValueError.
We use own Exception classes for this, upstream classes are deprecated
2018-08-06 17:32:55 +02:00
TW
4c8db8feb5
Merge pull request #4008 from ThomasWaldmann/fix-cachedir
don't archive tagged dir when recursing an excluded dir, fixes #3991
2018-08-06 17:29:52 +02:00
Thomas Waldmann
9654126247 minor change: simplify test setup
most of the files created via self.create_test_files() are not needed.
the only one is the empty file, so just create it here.

also: move repo init to after test file creation.
2018-08-06 14:28:46 +02:00
Thomas Waldmann
8256c26cf2 don't archive tagged dir when recursing an excluded dir, fixes #3991
if we are recursing an explicitly excluded directory, we are only
looking for explicitly included items, but we do not need to archive
tagged directories or the tag files themselves - they are all excluded
by the explicit exclusion already.

for performance reasons, we still determine whether a directory is
tagged and if it is, we do not recurse into it as there can't be any
included items in there.
2018-08-05 19:51:49 +02:00
Thomas Waldmann
d2e2f1b89d call socket.gethostname only once 2018-08-04 17:40:40 +02:00
Thomas Waldmann
90348c1de9 add BORG_HOST_ID, fixes #3985 2018-08-04 15:01:48 +02:00
Thomas Waldmann
52f1cd4bc6 argparse: work around bad default in py 3.7.0a/b/rc, fixes #3996 2018-07-30 01:04:14 +01:00
Thomas Waldmann
a7dd2560ba remove loggerDict.clear() from tearDown method, fixes #3805
It causes problems with the new caching in the py37 logger module.

Removing loggerDict.clear() fixes this and makes the tests work again.
Also, it does not seem to have any negative effect, neither on py36
nor on py37.

See also: https://bugs.python.org/issue34269
2018-07-29 12:29:26 +01:00
Thomas Waldmann
3417de85e0 locking: more logging
(cherry picked from commit f86bf8a0a8)
2018-07-17 19:06:45 +02:00
Thomas Waldmann
a51fca49bf process_alive: add some assertions
(cherry picked from commit 2bf1b29e8e)
2018-07-17 19:06:26 +02:00
TW
a7e408db61
Merge pull request #3980 from ThomasWaldmann/update-coafile
coala: exclude some more stuff, move ignores to "all" section
2018-07-17 01:16:22 +02:00
Thomas Waldmann
89c11f45ce cache lock: use lock_wait everywhere to fix infinite wait
also: clarify docs
(cherry picked from commit 2f3e60d9d5)
2018-07-16 23:50:04 +02:00
Thomas Waldmann
36cbc2d18f coala: ignore some stuff it warns about 2018-07-14 18:40:50 +02:00
Thomas Waldmann
e6fcf4ea42 update docs about separated compaction 2018-07-14 14:53:23 +02:00
Thomas Waldmann
a221ca16ad remote: introduce a dontcare flag (all values acceptable)
in some cases the value of a new parameter does not really matter,
as long as it works / there is no malfunction.

without this change, a new borg client could not commit to an old
borg server (and would fail late when trying to commit):

- when using compact=False as it does at most places

- when using cleanup_commits=True as a user might trigger by running
  borg compact --cleanup-commits

we never want a borg commit "fail late", leaving the repo in a state
with uncommitted data, thus the server shall just commit, no matter
how it handles compaction or whether it can do the cleanup.
2018-07-14 14:29:28 +02:00
Thomas Waldmann
3715d2da3e slightly refactor write_commit using new "want_new" flag 2018-07-14 14:29:28 +02:00
Thomas Waldmann
1f387d911a start new segment file for put/del to MANIFEST_ID
specialcase deleting / writing the manifest to be in a separate, new
segment file, so that when we supersede and compact it later, less
segment data has to be shuffled around - compaction can then just
delete this segment file and that's all.
2018-07-14 14:29:28 +02:00
Thomas Waldmann
755eaeec0a borg compact --cleanup-commits to get rid of leftover 17byte segments
see #2850.
2018-07-14 14:29:28 +02:00
Thomas Waldmann
efe60bd640 test fixes 2018-07-14 14:29:28 +02:00
Marian Beermann
aeef082483 repository: track commits in hints 2018-07-14 14:29:28 +02:00
Thomas Waldmann
de4afa097c separate borg compact command, fixes #2195 2018-07-14 14:29:28 +02:00
TW
d2a816d0d4
Merge pull request #3978 from ThomasWaldmann/fix-nanorst
nanorst: add missing blank to exception message
2018-07-13 11:38:52 +02:00
Thomas Waldmann
0d06407087 nanorst: add missing blank to exception message 2018-07-13 10:41:32 +02:00