upstream/borgbackup
1
0
Fork 0
mirror of https://github.com/borgbackup/borg.git synced 2026-02-20 00:10:35 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions 6
6873 commits 9 branches 150 tags 58 MiB
Commit graph

1924 commits

Author SHA1 Message Date
Thomas Waldmann
3fd5b73e1e borg create NAME ... 2022-06-20 20:17:28 +02:00
Thomas Waldmann
c085c2744b borg rename NAME NEWNAME 2022-06-20 15:18:24 +02:00
Thomas Waldmann
206245f3cd --repo: add -r short option 2022-06-16 15:41:37 +02:00
Thomas Waldmann
801ce819a3 help: archive name 2022-06-16 14:32:40 +02:00
Thomas Waldmann
3e765522de help: transfer from other repo 2022-06-16 14:23:47 +02:00
Thomas Waldmann
7dbf125083 Location: remove archive name 2022-06-16 14:18:47 +02:00
Thomas Waldmann
281bbbc16b fix tests and benchmarks 2022-06-16 14:18:39 +02:00
Thomas Waldmann
0f0cd24354 if --(other-)repo option is not given, use default from environment 
remove tests composing a repo+archive location with repo from env
and location from cli.
 2022-06-16 14:14:54 +02:00
Thomas Waldmann
1bf2a6a240 remove archive checks from location_validator, use --other-repo 2022-06-15 17:07:42 +02:00
Thomas Waldmann
1c707b7da2 cli: use --repo option instead of positional repo parameter 
currently still with ::archive appended.
 2022-06-15 16:36:56 +02:00
Thomas Waldmann
c01f3527e5 fix linter errors 2022-06-14 15:42:30 +02:00
Thomas Waldmann
f9132687e3 fix selftest count 2022-06-14 15:17:30 +02:00
Thomas Waldmann
e5ea016115 repository: set/query flags, iteration over flagged items (NSIndex) 
use this to query or set/clear flags in the "extra" word.

also: remove direct access to the "extra" word, adapt tests.
 2022-06-14 14:48:56 +02:00
Thomas Waldmann
bf9f42320e repository: sync write file in get_fd 
this fixes a strange test failure that did not happen until now:
it could not read the MAGIC bytes from a (quite new) segment file,
it just returned the empty string.

maybe its appearance is related to the removed I/O calls.
 2022-06-14 14:48:56 +02:00
Thomas Waldmann
3ce3fbcdff repository index: add payload size (==csize) and flags to NSIndex entries 
This saves some segment file random IO that was previously necessary
just to determine the size of to be deleted data.

Keep old one as NSIndex1 for old borg compatibility.
Choose NSIndex or NSIndex1 based on repo index layout from HashHeader.

for an old repo index repo.get(key) returns segment, offset, None, None
 2022-06-14 14:48:56 +02:00
Thomas Waldmann
75ad3b8e35 remove csize expectation from do_transfer upgrade_item 
the size is already remove via Item._update_internal method.
 2022-06-12 17:50:12 +02:00
Thomas Waldmann
822aefc7b0 re-add dsize placeholder 2022-06-12 17:15:13 +02:00
Thomas Waldmann
49adb77157 calc_stats: deduplicated size now, was deduplicated csize 
also: remove pre12_meta cache
 2022-06-12 17:15:13 +02:00
Thomas Waldmann
19dfbe5c5c compute the deduplicated size before compression 
so we do not need csize for it.
 2022-06-12 17:15:13 +02:00
Thomas Waldmann
1fd571a4d0 fix comments 2022-06-12 17:15:13 +02:00
Thomas Waldmann
2c1f7951c4 remove csize from ChunkIndexEntry 2022-06-12 17:15:13 +02:00
Thomas Waldmann
b82a39c3b3 remove csize from stats_against() 2022-06-12 15:48:33 +02:00
Thomas Waldmann
0211948cac remove csize from summarize return tuple 2022-06-12 15:48:33 +02:00
Thomas Waldmann
b726aa5665 remove csize support from get_size 2022-06-12 15:48:33 +02:00
Thomas Waldmann
ace5957524 remove csize from item.chunks elements 2022-06-12 15:48:33 +02:00
Thomas Waldmann
b9f9623a6d prepare to remove csize (set it to 0 for now) 2022-06-12 15:48:33 +02:00
Thomas Waldmann
d4ee968b07 use borg 2.0 to refer to this, not 1.3 
also, some type conversions are now done in update_internal once,
not in the decode methods of the classes in item.pyx.
 2022-06-09 18:13:40 +02:00
Thomas Waldmann
08228fbd32 Item: remove unused hardlink_masters param 2022-06-09 17:57:28 +02:00
Thomas Waldmann
58009f6773 Key: fix once, remove decode=... 2022-06-09 17:57:28 +02:00
Thomas Waldmann
ed22f721f3 EncryptedKey: fix once, remove decode=... 2022-06-09 17:57:28 +02:00
Thomas Waldmann
f2b085787b Item: disallow None value for .user/group/chunks/chunks_healthy 
If we do not know the value, just do not have that key/value pair in the item.
 2022-06-09 17:57:28 +02:00
Thomas Waldmann
64cc16a9f4 Item: fix xattr processing 
Item.xattrs is now always a StableDict mapping bytes keys -> bytes values.

The special casing of empty values (b'') getting replaced by None was removed.
 2022-06-09 17:57:28 +02:00
Thomas Waldmann
9d684120a2 Item: assert type also in property getter 
also: fixed Item.xattrs to be StableDict (not just a dict, as the
msgpack unpacker gives us)
 2022-06-09 17:57:28 +02:00
Thomas Waldmann
7b138cc710 Item: convert timestamps once, get rid of bigint code 2022-06-09 17:57:28 +02:00
Thomas Waldmann
8e58525fc6 Item: remove some decode= params 
update_internal() makes sure they have the desired type already.
 2022-06-09 17:57:28 +02:00
Thomas Waldmann
655c1b9cc2 update docstrings / comments 2022-06-09 17:57:28 +02:00
Thomas Waldmann
33444be926 more str vs bytes fixing 2022-06-09 17:57:28 +02:00
Thomas Waldmann
8e87f1111b cleanup msgpack related str/bytes mess, fixes #968 
see ticket and borg.helpers.msgpack docstring.

this changeset implements the full migration to
msgpack 2.0 spec (use_bin_type=True, raw=False).

still needed compat to the past is done via want_bytes decoder in borg.item.
 2022-06-09 17:57:28 +02:00
Thomas Waldmann
f8dbe5b542 cleanup msgpack related str/bytes mess, see #968 
see ticket and borg.helpers.msgpack docstring.
 2022-06-09 17:57:28 +02:00
Thomas Waldmann
c5540c2dd9 upgrade compressed chunk: fix treatment of ObfuscateSize chunks 
the inner payload of ObfuscateSize chunks are compressed chunks and need
the same zlib fix and level patching as non-obfuscated compressed chunks.
 2022-06-09 17:49:16 +02:00
Thomas Waldmann
72c68c49d0 obfuscation: fix byte order for size, fixes #6701 2022-06-09 17:49:16 +02:00
Thomas Waldmann
6584a92c81 compression: use the 2 bytes for type and level, fixes #6698 
adapt borg transfer, transferred chunks are set to compression level "unknown".
 2022-06-09 17:49:16 +02:00
Thomas Waldmann
32a3601e4a compute hlid from inode / device 2022-06-09 17:49:16 +02:00
Thomas Waldmann
8798b0340a use whitelist approach to make sure item._dict is clean 2022-06-09 17:49:16 +02:00
Thomas Waldmann
d3dfa3be30 use version 2 for new archives 
but still be able to read v1 archives
for borg transfer.
 2022-06-09 17:49:16 +02:00
Thomas Waldmann
e5f1a4fb4d recreate: cachedir_masters not needed any more 
now all hardlinked regular file items have chunks.
 2022-05-18 14:20:01 +02:00
Thomas Waldmann
6bfdb3f630 refactor hardlink_master processing globally 
borg now has the chunks list in every item with content.
due to the symmetric way how borg now deals with hardlinks using
item.hlid, processing gets much simpler.

but some places where borg deals with other "sources" of hardlinks
still need to do some hardlink management:
borg uses the HardLinkManager there now (which is not much more
than a dict, but keeps documentation at one place and avoids some
code duplication we had before).

item.hlid is computed via hardlink_id function.

support hardlinked symlinks, fixes #2379
as we use item.hlid now to group hardlinks together,
there is no conflict with the item.source usage for
symlink targets any more.

2nd+ hardlinks now add to the files count as did the 1st one.
for borg, now all hardlinks are created equal.
so any hardlink item with chunks now adds to the "file" count.

ItemFormatter: support {hlid} instead of {source} for hardlinks
 2022-05-18 14:20:01 +02:00
Thomas Waldmann
7903dad183 transfer: convert timestamps int/bigint -> msgpack.Timestamp, see #2323 
Timestamp scales to 64 or 96bit serialization formats, that should be enough for everybody.

We use this in archived items and also in the files cache.
 2022-05-18 14:20:01 +02:00
Thomas Waldmann
e4a97ea8cc transfer: all hardlinks have chunks, maybe chunks_healty, hlid 
Item.hlid: same id, same hardlink (xxh64 digest)
Item.hardlink_master: not used for new archives any more
Item.source: not used for hardlink slaves any more
 2022-05-18 14:20:01 +02:00
Thomas Waldmann
01f72d15b4 transfer: remove the zlib type bytes hack 
hack: see the docstring of ZLIB_legacy class.

New clean ZLIB class that works as every other compressor.

ZLIB ID 0x0500, ZLIB_legacy ID 0x.8..
 2022-05-18 14:20:01 +02:00