Commit graph

1396 commits

Author SHA1 Message Date
Thomas Waldmann
2211aaab48 fix crash when upgrading erroneous hints file, fixes #4922
if an old hints file gets converted to the new format and it
has entries referring to non-existent segment files, a crash
occurred.

with this code, the crash is avoided and the erroneous hints
entry is removed.
2020-03-09 00:08:39 +01:00
Thomas Waldmann
99f72beaa5 upgrade bundled zstd to 1.4.4 2020-03-08 21:39:06 +01:00
Thomas Waldmann
6a37af5f8b upgrade bundled lz4 to 1.9.2 2020-03-08 20:42:50 +01:00
Thomas Waldmann
8c532b9c73 ignore --stats when given with --dry-run, but continue, fixes #4373
... but continue the operation just as if --stats was not given.
2020-03-07 18:10:00 +01:00
Thomas Waldmann
2d20fa4e45 xxhash: remove xxh3 (does not compile)
we do not use that anyway.
2020-03-07 17:01:54 +01:00
Thomas Waldmann
96ff358887 upgrade xxhash to 0.7.3
https://github.com/Cyan4973/xxHash/releases/tag/v0.7.3

committed "as is", does not compile due to xxh3.
2020-03-07 17:01:42 +01:00
Will
9b3e28a41e Make flake8 happy about complex conditional 2020-03-05 12:50:28 -05:00
TW
5a28fc074d
Merge pull request #4980 from willsALMANJ/no-os-link-master
support platforms with no os.link (port to master)
2020-03-05 09:52:45 +01:00
TW
97fd0e5b57
Merge pull request #4975 from ThomasWaldmann/upgrade-xxhash
Upgrade xxhash (master)
2020-03-04 11:12:31 +01:00
TW
597b09a993 support platforms with no os.link (#4903)
support platforms with no os.link, fixes #4901

if we don't have os.link, we just extract another copy instead of making a hardlink.

for that to work, we need to have (and keep) the chunks list in hardlink_masters.
2020-03-03 23:34:54 -05:00
Thomas Waldmann
3905810e37 xxhash: remove experimental xxh3 code
it does not compile and we don't use it anyway.
2020-03-03 23:59:20 +01:00
Thomas Waldmann
810c60048b upgrade bundled xxhash to 0.7.2, fixes #4891
Looks like the minimum requirement for correct operations
on ARMv6 in non-fixup mode is xxhash 0.7.2.

note: this does not compile due to the experimental stuff in xxh3.h.
2020-03-03 23:58:54 +01:00
Thomas Waldmann
a8831f4978 fix ProgressIndicator msgids, fixes #4935
add some to code, fix docs.
2020-03-03 23:57:36 +01:00
Thomas Waldmann
fa63150e14 fix bug in hashindex_set on resize, fixes #4829
the problem was that after a resize that was triggered by too few
empty buckets, the rebuilt new hash table had entries at different
positions than before, but the idx where to SET the entry was not
recomputed afterwards.
2020-03-01 17:47:12 +01:00
Will
242424ecae Move sync_file_range to its own extension 2020-03-01 00:06:04 -05:00
Rémi Oudin
70c8538626 extract: fix confusing output of borg extract --list --strip-components, fixes #4934 2020-02-17 21:57:24 +01:00
Michael G. Noll
0d854e08e4 Document how to test exclusion patterns without performing an actual backup 2019-12-17 14:56:50 +01:00
Birkhoff Lee
5739602a11 Elaborate on "Calculating size" (#4866)
create: tell that "Calculating size" time and space needs are caused by --progress

It took me several days to figure out that `--progress` was the culprit to Borg being SIGKILL'd by the kernel, because of OOM, and because of I enabled progress option. This elaborates on what's the calculation for.
2019-12-15 23:05:40 +01:00
Rémi Oudin
c209eff842 Fix the test failing due to timestamp localisation, fixes #4853
from https://borgbackup.readthedocs.io/en/stable/usage/general.html#date-and-time

    Unless otherwise noted, we display local date and time.
    Internally, we store and process date and time as UTC.

OK, that does not directly say something about --timestamp, but as the cli is also part of the user interface (as output is), this might mean that maybe not the test needs fixing, but our processing of that option.

But, the docs also say:

    --timestamp TIMESTAMP
    manually specify the archive creation date/time (UTC,  yyyy-mm-ddThh:mm:ss format).
    Alternatively, give a reference  file/directory.

So, using UTC there is a documented feature, although it is different from output using localtime.

So, the test fix in this PR is correct according to the docs. \o/
2019-12-09 15:14:47 +01:00
TW
154239cd89
Merge pull request #4833 from ncleaton/chunkerloop
chunker speedup
2019-11-29 08:04:49 +01:00
Nick Cleaton
708fa83c1d chunker speedup and regression test 2019-11-28 20:59:02 +00:00
Rémi Oudin
a029d686b5 Borg recreate timestamp is a no op (#4815)
recreate: support --timestamp option, fixes #4745
2019-11-16 11:03:34 +01:00
Rémi Oudin
e393dd4fe1 Add a try catch when formatting the info string. (#4818)
info: if the archive doesn't exist, print a pretty message, fixes #4793
2019-11-16 10:56:03 +01:00
TW
1200690a3f
Merge pull request #4816 from jrast/fix-statvfs
Replaced usage of os.statvfs with shutil.disk_usage
2019-11-03 20:26:25 +01:00
Jürg Rast
f8856193d3 Replaced usage of os.statvfs with shutil.disk_usage
The RepositoryCache used `os.statvfs` which is not available on Windows.
`shutil.disk_usage` provides the same information but in a cross platform way.
2019-11-03 17:40:34 +01:00
Thalian
8a1b198e24 [DOC] #4674 - Add more documentation for @api decorator. (#4801)
docs: fix/improve documentation for @api decorator
2019-11-03 14:09:33 +01:00
Jürg Rast
b00559a5b9 Adding sys.path to %PATH% to find libcrypto on windows 2019-10-30 01:06:46 +01:00
Thalian
2209f56cd5 Feature/4674 compact threshold (#4798)
compact: add --threshold option, fixes #4674
2019-10-24 10:12:58 +02:00
TW
59ea5bc4d6
Merge pull request #4774 from ThomasWaldmann/fix-prefix-argparse-bug
--prefix / -P: fix processing, avoid argparse issue, fixes #4769
2019-10-19 21:43:10 +02:00
TW
008ee01ea9
Merge pull request #4773 from ThomasWaldmann/ignore-hardlink-eacces
ignore EACCES (errno 13) when hardlinking, fixes #4730
2019-10-19 21:40:19 +02:00
Thalian
212b438c3c [DOC] #4118 - Explain difference between --exclude and --pattern. (#4783)
docs: Explain difference between --exclude and --pattern, fixes #4118
2019-10-19 21:38:24 +02:00
TW
5ce902452b
Merge pull request #4779 from Gu1nness/issue-4752-build-man-is-broken
Fixes the man pages creation
2019-10-19 20:41:48 +02:00
Rémi Oudin
1d9b4d0159
Change variable name and remove useless variable 2019-10-19 12:33:26 +02:00
Björn Ketelaars
e2ee7fd24b Support msgpack-0.6.2.
This addresses
https://github.com/borgbackup/borg/issues/4360#issuecomment-541107988
2019-10-15 05:22:20 +02:00
Rémi Oudin
dd92695a04 Fixes the man pages creation
The issue https://github.com/borgbackup/borg/issues/4471 solution
created many commands for borgfs which shouldn't exist in addition to
creating issue https://github.com/borgbackup/borg/issues/4752.

Fixes https://github.com/borgbackup/borg/issues/4752 .
2019-10-09 15:14:53 +02:00
Thomas Waldmann
6395126d34 --prefix / -P: fix processing, avoid argparse issue, fixes #4769
changes:

- changed --prefix default to None (was: ''), so we can check using
"is not None" to determine when --prefix has been given.

- the previous check for --prefix being used was just for a truthy
value, so using --prefix='' was not really supported, but happened to
behave the same as the default processing anyway.

- argparse python stdlib code seems to have a bug when processing an
option like --prefix='--', args.prefix will be [] in that case (should
be '--'). With previous code this behaved like no prefix given ([] value
is not truthy). Now, as we check for "is not None", it will try to
process that value but blow up with a TypeError as it can't do [] + '*'.
This is a bit unpretty end, but at least borg prune won't delete all
your archives and it will be a reminder that argparse is broken.
- for borg check --repository-only, we also check for --glob-archives
not being used and give the warning otherwise.
2019-10-03 19:03:36 +02:00
Thomas Waldmann
851db7fe21 ignore EACCES (errno 13) when hardlinking, fixes #4730
we create the hardlink to be able to secure erase the old config file.

if we can't do that because there is just a problem with hardlinks not
working, the old config will be just overwritten normally (not secure
erased). the user will get a warning in that case, but other than that,
the overall borg operation will succeed.

if there is a bigger problem (like a general lack of permissions or a
general issue with the underlying fs), subsequent operations will fail.
2019-10-03 15:19:03 +02:00
Julien Lamy
cfa76e066b Add missing attribute to FUSE entries 2019-09-22 17:12:01 +02:00
TW
53f8882d31
Merge pull request #4751 from ThomasWaldmann/noatime-default
create: make --noatime the default, deprecate --noatime, fixes #4673
2019-09-06 23:08:38 +02:00
Thomas Waldmann
3f63234974 create: make --noatime the default, deprecate --noatime, fixes #4673
also: add --atime option to enable storing files' atime.
2019-09-06 21:48:54 +02:00
TW
aa7df50a2d
Merge pull request #4635 from ThomasWaldmann/ctrlc-checkpoint
first ctrl-c: checkpoint and abort, fixes #4606
2019-09-06 21:44:07 +02:00
TW
975f5627b1
Merge pull request #4736 from saurvs/forward-port-4729
new BORG_WORKAROUNDS mechanism, basesyncfile, fixes #4710
2019-08-27 21:55:58 +02:00
Thomas Waldmann
b43d3bb7fa new BORG_WORKAROUNDS mechanism, basesyncfile, fixes #4710
remove WSL autodetection. if WSL still has this problem, you need to
set BORG_WORKAROUNDS=basesyncfile in the borg process environment to
work around it.

(cherry picked from commit beb948fc71)
2019-08-27 19:37:40 +05:30
Thomas Waldmann
cb2d31ed98 fix partial extract for hardlinked contentless file types, fixes #4725
if the file is not a regular file, but a hardlink slave with a not
extracted hardlink master, chunks will be None and we must not call
preload(chunks).

(cherry picked from commit 291d58efa1)
2019-08-27 19:20:20 +05:30
Thomas Waldmann
9732fe4965 special behaviour on first ctrl-c, fixes #4606
like:
 - try saving a checkpoint if borg create is ctrl-c-ed
2019-08-25 22:49:09 +02:00
TW
373bd8abd3
Merge pull request #4696 from jrast/win10
WIP jrast/borg:win10, PR for better review and testing
2019-08-25 22:41:05 +02:00
Jürg Rast
bff97a99e1 Windows specific directory handling
On windows os.open does not work for directories.
If borg tries to open an directory on windows, None is returned
as file descriptor. The archive and archiver where adjusted to
handle the case if a file descriptor is None.
2019-08-24 10:17:18 +02:00
Jürg Rast
6b426d08d7 Initial work to build and run borg under windows
- Created a batch file to build borg on windows
- Adjusted setup.py to be runnable on windows and build the windows
extension
- Extracted the free space check to a function in the platform module
- Created the minimal needed (dummy) functions for the windows platform
module
2019-08-24 10:17:18 +02:00
ntova
627ca1376b Fix various code blocks in the docs (#4708)
Fix various code blocks in the docs

- rst markup: put codeblock markup on separate line to make it better visible / separate it from normal text colons.

- borg help texts in archiver.py: put codeblock markup instead of colon - that way it looks like a single colon when using the cli help and also works as code block markup.
2019-08-09 23:13:11 +02:00
TW
97eca3d287
Merge pull request #4713 from saurvs/forward-port-4698
Forward port #4698
2019-08-09 16:02:29 +02:00