Commit graph

748 commits

Author SHA1 Message Date
Marian Beermann
c2c90645ad Add Key.assert_id function 2016-08-14 17:41:33 +02:00
Marian Beermann
a80b371d09 Add decompress arg to Key.decrypt 2016-08-14 17:41:33 +02:00
Marian Beermann
ac80332ce7 Merge branch '1.0-maint' into master
# Conflicts:
#	src/borg/repository.py
#	src/borg/testsuite/repository.py
2016-08-14 15:51:49 +02:00
Thomas Waldmann
42b6a838da fix cyclic import issue, fix tests
needed to increase ChunkBuffer size due to increased items stream chunk size
to get the test working.
2016-08-14 15:26:56 +02:00
Thomas Waldmann
8be6761c26 Merge commit 'feb7e2517ef7ec07cc638953a86c726aada7d37e' 2016-08-14 15:05:11 +02:00
enkore
f8b2ffe999 Merge pull request #1458 from ThomasWaldmann/merge-1.0-maint
Merge 1.0 maint
2016-08-11 14:18:03 +02:00
enkore
006d1df517 Merge pull request #1450 from enkore/f/recreatefactors
minor recreate refactor
2016-08-09 23:28:58 +02:00
Marian Beermann
7d9fba5a41 ArchiveRecreater.copy_items: progress from target instead of source
(it produces the same output, just less confusing)
2016-08-09 21:20:13 +02:00
TW
ff5d9a3f12 Merge pull request #1447 from enkore/f/debug-topic
Debugging topics and some other minor stuff
2016-08-09 20:53:07 +02:00
Thomas Waldmann
c955d59273 Merge branch '1.0-maint' 2016-08-09 20:09:13 +02:00
Marian Beermann
0a65e83df3 check --verify-data: remove incorrect help paragraph
(since d0ec7e76bb - doesn't mean this probably useful feature may not
 be reimplemented, on the contrary)
2016-08-08 16:38:31 +02:00
Marian Beermann
bec5051ea5 helpers.format_timedelta: use timedelta.total_seconds() 2016-08-08 16:38:31 +02:00
Marian Beermann
389ca94407 Silence repeated "Processing files..." log entries when --no-files-cache 2016-08-08 16:38:31 +02:00
Marian Beermann
4d7a52d8e0 --debug-topic for granular debug logging
Especially: disables file-compression logging by default
2016-08-08 16:38:31 +02:00
Marian Beermann
7b70b74ad1 ArchiveRecreater.incref_partial_chunks try to use __contains__ 2016-08-08 15:54:47 +02:00
Marian Beermann
5fe6c09c34 Refactor ArchiveRecreater.try_resume 2016-08-08 15:54:41 +02:00
Thomas Waldmann
dbe862f3d8 Merge branch '1.0-maint' into merge-1.0-maint 2016-08-06 01:24:22 +02:00
TW
9b0d8df0f8 Merge pull request #1373 from PlasmaPower/filesystem-test-skipping
Filesystem feature detection and test skipping
2016-08-03 17:46:34 +02:00
TW
29e5e558bc Merge pull request #1412 from ThomasWaldmann/info-repo
borg info REPO
2016-08-02 23:29:39 +02:00
Thomas Waldmann
770a892b2d implement borg info REPO
currently it is just the same global stats also shown in "borg info ARCHIVE",
just without the archive-specific stats.

also: add separate test for "borg info".
2016-08-02 20:06:24 +02:00
enkore
b7a563885d Merge pull request #1413 from ThomasWaldmann/store-mountpoint-dirs
save mountpoint dirs, fixes #1033
2016-08-01 15:51:53 +02:00
Thomas Waldmann
2b454fc54b save mountpoint dirs, fixes #1033 2016-08-01 15:25:54 +02:00
Lee Bousfield
b86b5d952a
Filesystem feature detection and test skipping 2016-07-31 01:19:31 -04:00
Thomas Waldmann
9865509105 docs: remove borg list example, be more specific about compression heuristics 2016-07-30 21:21:45 +02:00
TW
02e229b255 Merge pull request #1377 from enkore/issue/1060
Use atomic file writing and updating for configuration and key files
2016-07-30 16:06:01 +02:00
Marian Beermann
e9a73b808f Check for sufficient free space before committing 2016-07-30 00:04:27 +02:00
Marian Beermann
389503db60 Repository.compact_segments: always save_space 2016-07-30 00:03:31 +02:00
Thomas Waldmann
999f0ae187 fix chunk refcounts considering part files 2016-07-28 19:39:29 +02:00
Thomas Waldmann
04ad1d1b0b use "part file", "part", etc. consistently
use .borg_part_N as filename to avoid collisions
2016-07-28 19:39:29 +02:00
Thomas Waldmann
a7fca52f07 create part items rather than checkpoint items
checkpoint items: chunks abc, abcdef, abcdefghi, ....
part items: chunks abc, def, ghi

solves the "restore a big file over unstable connection issue" (not very comfortably, but
one can just extract one part after the other and concatenate them to get the full file)
2016-07-28 19:39:29 +02:00
Thomas Waldmann
e5bd6cef20 add --consider-checkpoint-files option, update FAQ 2016-07-28 19:39:29 +02:00
Thomas Waldmann
49233be25d filter out checkpoint files
do not:
- list them
- extract them
- diff them
- include them for recreate
2016-07-28 19:39:29 +02:00
Thomas Waldmann
dd5f957e6f in-file checkpoints, fixes #1198, fixes #1093
also: unify code for stdin and on-disk file processing
2016-07-28 19:39:29 +02:00
Thomas Waldmann
75d91c4bd1 file chunking: refactor code a little
so it directly works with item.chunks list instead of a temporary list.
2016-07-28 19:39:28 +02:00
Thomas Waldmann
bda50b5958 stdin chunking: get Item into usable state early 2016-07-28 19:39:28 +02:00
Thomas Waldmann
9226fc6f6f split stat_attrs into cheap and expensive part
we already have stat results in st, so computing stat_simple_attrs is
rather cheap (except the username/groupname lookup maybe) and gets the
most important stuff right in the Item, so it is brought early into a
good state.

after chunking, stat_ext_attrs is called to add the more expensive-to-get
attributes, like bsdflags, xattrs and ACLs.
2016-07-28 19:39:28 +02:00
Marian Beermann
0ae48dafbb
ObjectNotFound: give ID as hex-string 2016-07-28 18:41:50 +02:00
Marian Beermann
d0ec7e76bb Fix borg-check --verify-data failing with rebuilt objects
There are some instances where --repair would do something. In these
instances --verify-data would fail if --repair was not given also, since
the changes without --repair are only in the index, but not in the
repository.
2016-07-28 18:41:08 +02:00
Marian Beermann
1b6b0cfae6 Fix borg-check --verify-data tripping over ObjectNotFounds 2016-07-28 18:40:20 +02:00
Lee Bousfield
b652f4039c
Remove trailing whitespace 2016-07-28 10:35:41 -04:00
Marian Beermann
1bad3fc552
Merge branch '1.0-maint' into master 2016-07-28 09:48:58 +02:00
Marian Beermann
ac8c8119a7
Merge branch '1.0-maint' into master 2016-07-28 00:29:43 +02:00
Marian Beermann
2e3fc9ddfc SyncFile/SaveFile: default binary=False, just like open() 2016-07-26 22:49:25 +02:00
Marian Beermann
863ab66908 SaveFile: unlink(tmppath): only ignore FileNotFoundError 2016-07-26 22:40:23 +02:00
Marian Beermann
dec671d8ff SaveFile: os.replace instead of rename 2016-07-26 22:39:45 +02:00
Lee Bousfield
ff108ef24b Catch a ValueError when checking if a user exists 2016-07-23 15:03:22 -04:00
Lee Bousfield
868586dd37 Fixed noatime detection 2016-07-23 13:05:35 -04:00
Lee Bousfield
88bffab935
Allow running tests outside of Vagrant
Fixes #1361
2016-07-23 10:31:16 -04:00
Marian Beermann
52a72cf73a Fix some item subscripts that came from 1.0-maint 2016-07-17 19:13:09 +02:00
TW
963bdd340c Merge pull request #1330 from enkore/issue/972
recreate: fix hogged signal handlers
2016-07-14 14:53:21 +02:00
Marian Beermann
fcb6930c53
recreate: fix hogged signal handlers 2016-07-14 02:11:13 +02:00
Marian Beermann
2e6eb63174
Merge branch '1.0-maint' into master 2016-07-14 01:28:26 +02:00
Thomas Waldmann
3baa8a3728 Merge branch '1.0-maint'
# Conflicts:
#	docs/changes.rst
#	docs/usage/mount.rst.inc
#	src/borg/archive.py
#	src/borg/archiver.py
#	src/borg/fuse.py
#	src/borg/testsuite/archiver.py
2016-07-11 01:23:27 +02:00
Marian Beermann
f4be2b3523 Use platform.SaveFile for repository, cache and key files
Fixes #1060
2016-07-09 21:10:46 +02:00
Marian Beermann
36ebc82748 Add platform.SaveFile 2016-07-09 21:10:26 +02:00
TW
4f7a211b02 Merge pull request #1290 from enkore/issue/1287-bp
sync_dir: silence fsync() failing with EINVAL
2016-07-08 18:31:26 +02:00
Marian Beermann
567617ebbe
sync_dir: silence fsync() failing with EINVAL 2016-07-08 18:05:46 +02:00
TW
65475ea1aa Merge pull request #1286 from enkore/issue/1269
Remove InternalOSError
2016-07-08 14:00:07 +02:00
Marian Beermann
6c3ca63456
Remove InternalOSError
obsoleted by 5b453856ec and e96905c6b
2016-07-08 13:35:14 +02:00
Marian Beermann
11e02461d2
1.0-maint merge fixup (hexlify..decode -> bin_to_hex) 2016-07-08 13:29:51 +02:00
Marian Beermann
aade62c9f9
wrap get_flags with backup_io 2016-07-08 13:26:10 +02:00
Thomas Waldmann
fe627246e4 Merge branch '1.0-maint' 2016-07-08 12:13:52 +02:00
Thomas Waldmann
12f55f4e9f Merge branch '1.0-maint' 2016-07-06 01:33:53 +02:00
Thomas Waldmann
f363ddd7ca Merge branch '1.0-maint' 2016-07-04 20:11:21 +02:00
enkore
69f73b1752 Merge pull request #1227 from ThomasWaldmann/checkpoint-interval
increase checkpoint-interval default from 300 to 1800s, fixes #896
2016-06-30 17:09:56 +02:00
Thomas Waldmann
87d6755108 Merge branch '1.0-maint' 2016-06-29 18:28:33 +02:00
Thomas Waldmann
c201654f45 increase checkpoint-interval default from 300 to 1800s, fixes #896
doing a checkpoint can be expensive, if the repo connection is slow and/or
accessed via a network filesystem and/or the repo is large.

thus: don't do it too often.
2016-06-27 20:05:20 +02:00
Thomas Waldmann
5791b89c04 use bytes type for ACL attributes, fixes #1205
this saves some encoding / decoding dance compared to having a str interface.
2016-06-26 23:04:04 +02:00
Thomas Waldmann
323894d195 Merge branch '1.0-maint' 2016-06-26 20:46:12 +02:00
Thomas Waldmann
ed128e517b Merge branch '1.0-maint' 2016-06-25 17:18:14 +02:00
TW
3ff6ac09a6 Merge pull request #1179 from Abogical/list-format
Apply --format when listing repository archives, fixes #1166
2016-06-24 18:29:03 +02:00
Abogical
6658112f8c Add documentation for listing repository archives 2016-06-21 20:01:01 +03:00
Abogical
87173d25de Added testsuite for listing repository archives 2016-06-20 19:23:22 +03:00
Abogical
c839eb63f7 Apply --format when listing repository archives 2016-06-20 19:23:15 +03:00
Abogical
0942bcd1b4 Move code block for the write() function at do_list() 2016-06-20 16:14:52 +03:00
James Clarke
d4f5172771 Correctly handle multiple LD_PRELOAD entries; fixes #1111 2016-06-19 22:04:03 +01:00
TW
d9a14e3658 Merge pull request #1184 from c4rlo/linux-syncfile-errno-fix
platform.linux: small SyncFile error handling fix
2016-06-18 14:49:18 +02:00
Carlo Teubner
58d236c619 platform.linux: small SyncFile error handling fix 2016-06-18 13:39:22 +01:00
Carlo Teubner
ff470f49f3 typo fixes in comment+docstring 2016-06-18 12:57:54 +01:00
Thomas Waldmann
9a64835b4d Merge branch '1.0-maint'
Also: add missing keys to ARCHIVE_KEYS set.
2016-06-13 00:14:08 +02:00
TW
8c26945097 Merge pull request #1158 from ThomasWaldmann/pretty-key
add Key/EncryptedKey PropDict, see #1157
2016-06-12 14:51:46 +02:00
Thomas Waldmann
d2468d37df add Key/EncryptedKey PropDict 2016-06-12 04:28:40 +02:00
Michael Herold
d45a13babf Changes archive format from uuid to uuid4
This allows for future support of other uuid versions
2016-06-11 22:57:03 +02:00
Michael Herold
5aa97e283c Adds uuid to archive format tags 2016-06-11 12:03:55 +02:00
jenkins
b86775ab4f
Fix test_sparse_file for solaris 2016-06-10 12:21:03 +02:00
Thomas Waldmann
d813775065 Merge branch '1.0-maint' 2016-06-09 20:29:44 +02:00
Marian Beermann
e41f3ecc5b
SyncFile: open with O_EXCL (or equivalent) 2016-06-04 22:47:27 +02:00
Thomas Waldmann
75244ea6d1 fuse: simplify timestamp processing 2016-06-04 17:34:03 +02:00
Thomas Waldmann
60da32123a refactor to use Item class, fixes #1071 2016-06-04 17:24:55 +02:00
Marian Beermann
0ca1a18b33
Make borg.platform a package 2016-05-31 15:38:56 +02:00
Marian Beermann
0333f91eb2 platform: rearrange imports 2016-05-31 15:38:16 +02:00
Marian Beermann
2d90bb2c55 Compose platform_base from the platform-specific functionality
Noticed while working on the windows branch.
2016-05-31 15:38:16 +02:00
Thomas Waldmann
d3deec9fc9 cleanup / reorder imports
import stdlib
from stdlib import x

import 3rd party

import borg stuff
2016-05-31 01:18:03 +02:00
Thomas Waldmann
49484d6d67 split helpers import into multiple ones 2016-05-31 00:33:13 +02:00
Thomas Waldmann
7e0ea1635a Merge branch 'master' into move-to-src 2016-05-30 19:41:30 +02:00
Thomas Waldmann
045e5a1203 Merge branch 'master' into move-to-src 2016-05-30 19:38:16 +02:00
Thomas Waldmann
3ce35f6843 Merge branch 'master' into move-to-src 2016-05-21 19:06:01 +02:00
Thomas Waldmann
d1ea925a5b move borg package to src/ 2016-05-05 20:19:50 +02:00