Commit graph

320 commits

Author SHA1 Message Date
TW
21f26988cc Merge pull request #303 from anarcat/verbosity
Verbosity
2015-10-19 22:25:46 +02:00
Antoine Beaupré
38bde26673 remove needless double type conversion
i got confused and thought i was setting the environment (in which case you need to cast to str anyways, so this was wrong even then)
2015-10-19 15:47:05 -04:00
Antoine Beaupré
a3084b7174 try to fix build on py32 again 2015-10-19 14:41:02 -04:00
Antoine Beaupré
12d22d5940 try to fix build on py32 2015-10-19 09:10:45 -04:00
Antoine Beaupré
f9bccd1987 revert policy decision of showing all files listing only on debug 2015-10-19 09:10:19 -04:00
Antoine Beaupré
49d7c240a0 cosmetic: don't word-wrap needlessly 2015-10-19 09:10:04 -04:00
Thomas Waldmann
7d31da7914 implement cmdline metadata value decoding
code copied & adapted from Archive.load()
2015-10-19 12:16:28 +02:00
Thomas Waldmann
af2f1f7861 remove a fixed FIXME
see a few lines below, where it renames the original file to *.beforerecover before creating the repaired copy using the old name.
2015-10-19 11:57:12 +02:00
TW
79482a1c46 Merge pull request #310 from ThomasWaldmann/non-ascii-acls
Non ascii acls
2015-10-19 11:39:15 +02:00
TW
a7db210ee9 Merge pull request #308 from ThomasWaldmann/test-binary
Test binary, fixes #215
2015-10-19 10:14:20 +02:00
Antoine Beaupré
98512736e5 stats tests: vary values to possible catch errors 2015-10-18 21:13:01 -04:00
Antoine Beaupré
f48bbc3725 move defaults up in alternate implementation, use nameduple 2015-10-18 21:07:28 -04:00
Antoine Beaupré
158e6b529a cosmetic: pep8 and io is actually just out 2015-10-18 21:06:56 -04:00
Antoine Beaupré
4c6915cce5 fix typos and remove debug code 2015-10-18 21:06:37 -04:00
Antoine Beaupré
1c0fb82b59 check for the stream --progress uses, not stdin 2015-10-18 20:54:39 -04:00
Thomas Waldmann
d602558f9d acls (darwin): use helper functions safe_encode/safe_decode
also: rewrite one bytes-based function to work more similar than all other acl-processing functions.
2015-10-19 02:32:29 +02:00
Thomas Waldmann
213075e5c1 acls (freebsd): use helper functions safe_encode/safe_decode 2015-10-19 01:43:46 +02:00
Thomas Waldmann
259f8678d6 acls (linux): helper functions safe_encode/safe_decode 2015-10-19 01:40:15 +02:00
Thomas Waldmann
5a2aa7b02b acls (linux): make tests as they should be
TODO: can this be done without creating a user/group "übel" with uid 666 gid 666?
2015-10-19 01:09:58 +02:00
Thomas Waldmann
e818026440 acls (linux): remove int() cast
it seems like the (non-numerical) name or group might get into fields[3] if the name -> uid or group -> gid mapping is unknown.
2015-10-19 00:28:53 +02:00
Thomas Waldmann
1f14d1de19 acls (linux): use surrogatescape error handling for acl_append_numeric_ids and acl_numeric_ids
surrogatescape will decode/encode invalid utf-8 sequences (if we do not get utf-8) in a round-tripping way.
2015-10-19 00:01:19 +02:00
Thomas Waldmann
63ed5d9949 acls (posix platforms): use surrogatescape error handling for posix_acl_use_stored_uid_gid
surrogatescape will decode/encode invalid utf-8 sequences (if we do not get utf-8) in a round-tripping way.
2015-10-18 23:43:24 +02:00
Thomas Waldmann
b2cffe00fe acls (linux): use surrogatescape error handling, fix test
surrogatescape will decode/encode invalid utf-8 sequences (if we do not get utf-8) in a round-tripping way.
2015-10-18 23:33:29 +02:00
Thomas Waldmann
55179fe64d acls (linux): add a test for acl_use_local_uid_gid and fix this function
this has never worked as intended as the function was not using the computed "fields[1]" value at all.
plus there were type issues after that was fixed.
2015-10-18 23:20:01 +02:00
Thomas Waldmann
45b04cd3c1 add non-ascii ACL test (linux)
still failing as the tested code does not yet work with non-ascii ACLs
2015-10-18 20:54:46 +02:00
Thomas Waldmann
723590df3d fix borg.exe presence detection for python3.2 2015-10-18 19:18:24 +02:00
Thomas Waldmann
60cdb5f716 make archiver tests run for the python source as well as for the pyinstaller binary, fixes #215
if the borg.exe binary is not available in PATH, binary tests are skipped.

source tests are run without forking (for better speed, esp. on travis).
binary tests need forking the binary, of course.

for source tests, some tests check for an exception to happen.
for a forked binary, we of course can only check the exit code, which is non-zero in that case.
2015-10-18 18:48:36 +02:00
Thomas Waldmann
03975016c5 upgrader: fix / reformat comments 2015-10-18 16:43:59 +02:00
TW
3c52f41132 Merge pull request #299 from anarcat/no-inplace
do not upgrade repositories in place by default
2015-10-18 16:36:07 +02:00
TW
10567fb327 Merge pull request #306 from anarcat/upgrader-index-fixes
fix cascading failure with the index conversion code
2015-10-18 14:54:49 +02:00
Antoine Beaupré
319fcbacc5 fix phrasings in "in place" refactoring 2015-10-17 22:04:00 -04:00
Antoine Beaupré
8c48015e70 try to work around Windows idiocy of open files
apparently, we may be able to rewrite an open file if we rename it first.

i have no way to test this, unfortunately
2015-10-17 21:59:51 -04:00
Antoine Beaupré
e59f1d50d9 fix tests: umask is not defined in the repo yet 2015-10-17 21:56:25 -04:00
Antoine Beaupré
ce72051284 rename cache variable to index for clarity 2015-10-17 21:50:16 -04:00
Antoine Beaupré
07e7f2dcad fixup warning message 2015-10-17 21:50:14 -04:00
Antoine Beaupré
fb68c6ff70 always convert the chunks cache we find
it seems it is possible that the chunks files are copied but *not*
converted. this may have happened here because the conversion was
interrupted, although the specific scenario is still unclear (but it
did happen during manual tests here). therefore reproducing this
problem seems to be difficult, hence the lack of tests for this
specific issue.

since we consider the header replacement code to be safe, that we
always convert shouldn't pose any additional threat to the existing
borg chunk cache.
2015-10-17 21:44:50 -04:00
Antoine Beaupré
c4599d8ea4 fix cascading failure with the index conversion code
this resolves bug #something where the index file could not be
converted, completely breaking conversion.

it seems that, during some refactoring, the index conversion code was
completely dropped. this was missed by the unit tests because the repo
can still be opened by the constructor even though the index is
invalid, so tests need improvements there.
2015-10-17 21:44:50 -04:00
Antoine Beaupré
92ac120fb0 default progress display to true if on a tty
this makes --progress a toggle: if there's a terminal, it turns it
off, if there isn't, it forces it on.
2015-10-17 21:21:33 -04:00
Antoine Beaupré
e4f325182e Revert "no need to flush stderr in my tests"
It is actually necessary, for now.

This reverts commit 8fdd1eddf9.

Conflicts:
	borg/helpers.py
2015-10-17 21:21:33 -04:00
Antoine Beaupré
ce1aaa9dfa Revert "fix conflict between --stats and --progress"
the columns handling fixed that isssue more elegantly

This reverts commit 7f77778419.

Conflicts:
	borg/helpers.py
2015-10-17 21:21:33 -04:00
Antoine Beaupré
0d8525ad8f add missing lines 2015-10-17 21:21:33 -04:00
Antoine Beaupré
d666c86bfc adjust display of --stats
it was broken by recent commits.

also remove the __format__() anti-pattern from cache as well.
2015-10-17 21:21:33 -04:00
Antoine Beaupré
28d0a9ce84 human-readable representation of stats 2015-10-17 21:21:32 -04:00
Antoine Beaupré
66bfc6fce8 get rid of print_() function
this function was over-coupling the cache system and the statistics
module. they are now almost decoupled insofar as the cache system has
its own rendering system now that is called separately.

furthermore, we have a much more flexible formatting system that is
used coherently between --progress and --stats

the degenerate case here is if we want to change the label in the
statistics summary: in this case we need to override the default
__str__() representation to insert our own label.
2015-10-17 21:21:32 -04:00
Antoine Beaupré
118ee8679f remove custom microformat from stats
i saw the errors in my ways: __format__ is only to customize the
"format mini-language", what comes after ":" in a new string
format. unfortunately, we cannot easily refer to individual fields in
there, short of re-implementing a new formatting language, which seems
silly.

instead, we use properties to extract human-readable versions of the
statistics. more intuitive and certainly a more common pattern than
the exotic __format__().

also add unit tests to prove this all works
2015-10-17 21:21:32 -04:00
Antoine Beaupré
6b547e6554 no need to flush stderr in my tests 2015-10-17 21:21:32 -04:00
Antoine Beaupré
75c993b875 simplify progress display
we stop enforcing a minimum width for fields, it changes only on
logarithmic boundaries, so not a big problem. string conversion is
implicit

this gives us a little more width for the path
2015-10-17 21:21:32 -04:00
Antoine Beaupré
ecae163072 use all available columns for path in progress
we use the new get_terminal_size() function, with a fallback for
Python 3.2. we default to 80 columns.

then we generate the stats bit and fill the rest with the path, as
previously, but with a possibly larger field.

note that this works with resizes in my test (uxterm)
2015-10-17 21:21:32 -04:00
Antoine Beaupré
7f28244cfe fix conflict between --stats and --progress
the --stats output would be slightly garbled by --progress, because of
the \r that is output at the last line...

example:

    initializing cache
    reading files cache
    processing files
    ------------------------------------------------------------------------------ s/twotone
    Archive name: 2015-10-15-test
2015-10-17 21:21:32 -04:00
Antoine Beaupré
e8cf28f9a0 only show all files found in debug
a single -v flag shouldn't flood the console with all the files in the
path specified, it makes -v basically useless

this way, -v can also be used with --progress to have nicer output:

initializing cache
reading files cache
processing files
  5.20 GB O   2.66 GB C  25.13 MB D 27576 N baz/...
2015-10-17 21:21:31 -04:00