Commit graph

1617 commits

Author SHA1 Message Date
TW
35d2587aa6
Merge pull request #5860 from braewoods/verify_data
resolve clarify borg check --verify-data (#5808)
2021-06-19 14:08:00 +02:00
James Buren
4f9e30e980 resolve clarify borg check --verify-data (#5808) 2021-06-18 15:00:14 -05:00
James Buren
7b56105424 add paragraph added in #5855 to utility documentation source 2021-06-18 14:16:56 -05:00
Thomas Waldmann
506c01dc8f import-tar: fix empty user/group name in TarInfo, fixes #5853
if the tar has no information about user/group name (empty string),
we must assign None to Item.user/group (not the empty string).
2021-06-17 15:59:41 +02:00
Thomas Waldmann
b15469c3c8 drop support for py36, require py37+, fixes #5790
if you do not have python >= 3.7, you can still use borg 1.1.x or 1.0.x.

another option is to use the fat binary from github releases, which
includes python and all other stuff needed.
2021-06-16 15:31:34 +02:00
TW
da7dd388ca
Merge pull request #5842 from braewoods/master
create --dry-run: fix display of kept tagfile, fixes #5834
2021-06-16 15:07:51 +02:00
Thomas Waldmann
4572974218 fix missing parameter in "did not consistently fail" msg, see #5822 2021-06-15 23:10:37 -05:00
Thomas Waldmann
243135167a create --dry-run: fix display of kept tagfile, fixes #5834 2021-06-15 18:29:58 -05:00
Thomas Waldmann
de84440c18 use context manager for filter process setup/teardown 2021-06-14 16:03:49 +02:00
Thomas Waldmann
9d23ccbf63 move tar open/close closer to iteration 2021-06-14 16:03:49 +02:00
Thomas Waldmann
b0af91837d minor fixes 2021-06-14 16:03:49 +02:00
Thomas Waldmann
ab1757edab import-tar: better test 2021-06-14 16:03:49 +02:00
Thomas Waldmann
c84895a5eb dedup get_tar_filter code
also:
- check for short tarfile extensions
- add .lz4 and .zstd
- fix docs about extensions and decompression commands
2021-06-14 16:03:20 +02:00
Thomas Waldmann
d4d18e851b progress / stats cleanup 2021-06-14 15:37:58 +02:00
Thomas Waldmann
fb2efd88fe implement TarfileObjectProcessors similar to FilesystemObjectProcessors 2021-06-14 15:37:58 +02:00
Thomas Waldmann
5304693c55 import-tar: support fifos and blk/chr devices 2021-06-14 15:37:58 +02:00
Thomas Waldmann
5a3d9db4f7 add a simple test for import-tar 2021-06-14 15:37:58 +02:00
Thomas Waldmann
8df97f5016 rename create-from-tar -> import-tar
we already have export-tar, so import-tar is a better corresponding name.

also: improve docs
2021-06-14 15:37:46 +02:00
Thomas Waldmann
9fc350ff20 create-from-tar: fix and adapt to current master branch
- remove --progress (it is a common option now)
- remove --compression-from COMPRESSIONCONFIG (not supported any more)
- chunker-params: ALGO added
- return exit code, fix compression param
- give compatibility argument to with_repository decorator
- fix typos / c&p errors
- remove duplicate archive.add_item() call
- use dash_open(), this also fixes: use sys.stdin.buffer (binary)
- remove non-working compression/chunking code,
  this needs a bigger refactor...
2021-06-09 22:52:38 +02:00
Marian Beermann
c14197e559 borg create-from-tar 2021-06-09 22:52:38 +02:00
Elmar Hoffmann
938e7f295c add progress indicator for archive check
Depending on the number of archives in a repository, the archive check part
of the check operation can take some time, so it should have a progress
indicator as well.
2021-05-15 23:15:31 +02:00
Thomas Waldmann
b04b7a77fd fix --iec option when given after "borg"
now both behave the same:
borg --iec info REPO
borg info --iec REPO
2021-05-03 18:28:58 +02:00
TW
c114e060ec
Merge pull request #5788 from RonnyPfannschmidt/move-chunks-equals
move chunk_equals to module level and modernize tox.ini
2021-05-02 21:14:59 +02:00
Ronny Pfannschmidt
603023bbd5 transform _chunk_content_equal into a global function to ease later benchmarking 2021-05-02 17:29:37 +02:00
Thomas Waldmann
65ea375968 get rid of distutils, use packaging 2021-05-01 20:40:48 +02:00
Thomas Waldmann
76dfd64aba create/recreate: print preliminary file status early, fixes #5417
if we back up stdin / pipes / regular files (or devices with --read-special),
that may take longer, depending on the amount of content data (could be many GiBs).

usually borg announces file status AFTER backing up the file,
when the final status of the file is known.

with this change, borg announces a preliminary file status before
the content data is processed. if the file status changes afterwards,
e.g. due to an error, it will also announce that as final file status.
2021-04-30 20:34:13 +02:00
Romain Vimont
9ddcfaf4f7 info / create --stats: add --iec option
If --iec is passed, then sizes are expressed in powers of 1024
instead of 1000.
2021-04-28 15:17:40 +02:00
Thomas Waldmann
dec1664a7e missing / healed chunks: always tell chunk ID, fixes #5704 2021-04-19 23:46:21 +02:00
Thomas Waldmann
02a9db50d2 do not load files cache for commands not using it, fixes #5673 2021-04-19 22:40:21 +02:00
Thomas Waldmann
adc89c488c fix repeated cache tag file writing bug
this bug was introduced by PR #5485 and also affects backport PR #5774.
2021-04-19 21:07:29 +02:00
Thomas Waldmann
7290333051 help string fixup 2021-04-16 19:09:52 +02:00
Thomas Waldmann
6b838febe8 s/remote_(buffer|ratelimit)/upload_(buffer/ratelimit)/g 2021-04-16 15:48:10 +02:00
Thomas Waldmann
40e4aa085f add --upload-(ratelimit|buffer), deprecate --remote-*, fixes #5611 2021-04-16 15:45:10 +02:00
Thomas Waldmann
6f9b9e5a53 s/numeric_owner/numeric_ids/g 2021-04-16 15:02:16 +02:00
Thomas Waldmann
251ce11fd8 add --numeric-ids, deprecate --numeric-owner option, fixes #5724 2021-04-16 14:52:12 +02:00
adrian5
6f371d5522 docs: Add commas to pattern help section 2021-03-19 09:34:51 +01:00
Robert Blenis
b2dea4422e add --json-lines option to diff command 2021-03-13 11:50:55 -05:00
Thomas Waldmann
bbccdbd81c mount: implement --numeric-owner (default: False!), fixes #2377
this is different default behaviour than in borg < 1.2:

default (numeric_owner=False) is to use the user/group name from the archive,
look up the local uid / gid and then use that for the FUSE fs.

when --numeric-owner is given (numeric_owner=True), then the uid/gid
from the archive is directly used (as it was the default behaviour in
borg < 1.2).

this was implemented like this (changing the default behaviour) to make
borg mount and borg extract behave more similar considering usage of
user/group numeric archived ids or archived names mapped to corresponding
numeric local system ids.

also, both now use the same function to get the uid/gid from the item.

fuse:
- add user and group name entries to default_dir
- also: set internal_dict(!) of new Item with data from Item.as_dict()
2021-03-07 18:16:23 +01:00
TW
086c0f2a6e
Merge pull request #5713 from ThomasWaldmann/files-cache-debug-logging-master
verbose files cache logging via --debug-topic=files_cache, fixes #5659
2021-02-28 23:14:28 +01:00
Thomas Waldmann
2211b840a3 verbose files cache logging via --debug-topic=files_cache, fixes #5659 2021-02-28 22:39:44 +01:00
Thomas Waldmann
d44836a865 config: accept non-int value for max_segment_size
borg config REPO max_segment_size 500M

note: when setting a non-int value for this in a repo config, using the repo will require borg >= 1.1.16.
2021-02-28 22:28:58 +01:00
Thomas Waldmann
99aa15b850 config: accept non-int value for storage_quota
borg config REPO storage_quota 100G

note: when setting a non-int value for this in a repo config, using the repo will require borg >= 1.1.16.
2021-02-28 22:27:48 +01:00
Thomas Waldmann
d4971e2819 some micro-opts in stat_ext_attrs 2021-02-16 23:24:05 +01:00
Thomas Waldmann
1b65db990d create/extract: add --noxattrs option, #3955
when given with borg create, borg will not get xattrs from input files (and thus, it will not archive xattrs).

when given with borg extract, borg will not read xattrs from archive and it will not set xattrs on extracted files.
2021-02-16 23:20:28 +01:00
Thomas Waldmann
9412a8430e create/extract: add --noacls option, #3955
when given with borg create, borg will not get ACLs from input files (and thus, it will not archive ACLs).

when given with borg extract, borg will not read ACLs from archive and it will not set ACLs on extracted files.
2021-02-16 22:43:08 +01:00
Manu
a84ead8e7c Pass args.log_json to FilesystemObjectProcessors/Statistics instance 2021-02-07 10:42:46 +08:00
Thomas Waldmann
10368f9c2c minor fix: fix order issue in test 2021-02-06 01:24:14 +01:00
Thomas Waldmann
3d0c61a184 revert incorrect fix for put updating shadow_index, fixes #5661
A) the compaction code needs the shadow index only for this case:

segment A: PUT x, segment B: DEL x, with A < B  (DEL shadows the PUT).

B) for the following case, we have no shadowing DEL (or rather: it does not matter,
because there is a PUT right after the DEL) and x is in the repo index,
thus the shadow_index is not needed for the special case in the compaction code:

segment A: PUT x, segment B: DEL x PUT x

see also PR #5636.

reverts f079a83fed
and clarifies the code by more comments.

we keep the code deduplication of 5f32b5666a
and just add a update_shadow_index param to make it not look like there was
something accidentally forgotten, which was the whole reason for the reverted
"fix".
2021-02-04 02:29:43 +01:00
Thomas Waldmann
a77db94b01 fix bad default: manifest.archives.list(consider_checkpoints=False), fixes #5668
also, add a comment about it, to avoid future similar mistakes.
2021-02-03 14:13:27 +01:00
Thomas Waldmann
649603f247 only print stats if not Ctrl-C'ed
if the user interrupts by ctrl-c, we do not save the archive,
thus we can not show stats because archive.id will not be in
the chunks index.
2021-02-03 01:56:52 +01:00